When utilizing a UI abstraction, then the data you're displaying is protected from implementation changes in the UI layer. Does/should/can this extend to higher-level things, like for example, display as a tree or a grid? I can't work out how to insulate the abstraction from the higher-level details of how the UI is going to display the data garnered through said abstraction.
You want to start with the Model-View-Controller architecture. This allows you to insulate, as much as possible, your user interface from data changes. The Model layer is your data objects. The View layer is your actual Swing components. The Controller layer is your listeners. The Model layer is written independently of the other two, with no knowledge of the classes. The View layer is written with no knowledge of the Controller layer.
If you require more abstraction than that, you can create interfaces for your Model layer so that several different data models can all use the same interface. This way, it doesn't matter what data you give to the View layer, it just displays it through the use of the interface.
Also realize that it's not always possible to do what you're asking. Sometimes a user interface needs to be written specifically for the data being displayed. A tree isn't always a tree, and a grid isn't always a grid. It works well to customize the View layer to match the data being displayed. This way, you can tailor the functionality specifically to the data being manipulated and create a better interface for your users.
Certainly, though, it should be done where it makes sense. This is where experience and judgment play a big factor.