It doesn't do pooling like SqlConnection, if that is what you mean.
[caveat: I use "connection" here loosely to mean a logical connection, not necessarily a physical connection]
Between using a connection on-demand, and keeping one around, there are advantages and disadvantages to both approaches. There is some overhead in initializing a connection, so if you are doing 5 things you should try to do them on the same proxy - but I wouldn't keep a long term proxy around just for the sake of it.
In particular, in terms of life-cycle management, once a proxy has faulted, it stays faulted - so you need to be able to recover from the occasional failure (which should be expected). Equally, in some configurations (some combinations of session/instancing), a connection has a definite footprint on the server - so to improve scalability you should keep connections short-lived. Of course, for true scalability you'd usually want to disable those options anyway!
The cost of creating a connection also depends on things like the security mode. IIRC, it will be more expensive to open a two-way validated connection using message security than it will to set up a TransportWithMessageCredential connection - so "YMMV" is very much the case here.
Personally, I find that the biggest common issue with proxy performance isn't anything to do with the time to set up a conncetion - it is the chattiness of the API. i.e.
Scenario A:
- open a connection
- perform 1 operation with a large payload (i.e. a message that means "do these 19 things")
- close the proxy
Scenario B:
- open a connecton
- perform 19 operations with small payloads
- close the connection
Then scenario A will usually be significantly faster due to latency etc. And IMO don't even think about distributed transactions over WCF ;-p