I think it's entirely possible, but it would mean a different take on the server side of things.
If you do interface-based POJO development of the server-side components that the UI communicates with to do the work, you leave yourself the option of either an in-memory implementation or one that acts as a proxy to a remote component. The Swing desktop UI might use the in-memory version of the server-side, while the GWT version talks to the distributed version.
If you use Spring, it's easy to inject the appropriate implementation for each UI. Both only know about the interface and are oblivious to the implementation details.
I imagine a single code base that might be packaged in different ways. The Swing desktop app would be packaged and delivered as a JAR file. It wouldn't include the GWT or web tier controller classes. The GWT code base would include web configuration and controllers, because controllers are really part of the web tier.
The service interface and everything behind it would remain the same for both. The only difference would be that the Swing desktop configuration would inject the in-memory service layer implementations of the service interface. The web configuration would inject the web-based implementations of the service interface.
Whether my attempt to read your mind for requirements is correct or not, this approach has value regardless of how you decide to implement the service and client tiers.
I'm a Spring fan, so I'd recommend the use of Spring.