This is an area of Java programming that is highly under-documented. As you mention, extending from JFrame or JDialog to develop a GUI is not a good design practice, yet you see it all over the place in sample code.
JSR 296 is a useful starting place, but it's architecture has some serious problems. I do use JSR 296, but I have my own flavor of it, and consistently have to work around issues introduced by the framework design.
I've long thought that there should be a discussion group/wiki/something focused on this topic. So far, I've found the listserv for various rich client libraries to be useful, but not comprehensive. Something to think about starting maybe, in my free time :-)
So I can't provide any definitive resources for best practices in building swing applications. But I can give you some pointers to the toolkits and concepts that I've found that I use over and over again. Maybe these will be useful to you as you get going. Also, if enough people are interested in having a discussion about best practices, sharing code, etc... I'd be interested in being part of it.
First, some absolutely critical libraries if you are going to do Swing development:
Binding - there are a number of libraries that do this (JGoodies, JSR295 which has spun off into an open source project called Better Beans Binding (BBB), Eclipse binding framework). I started years ago using JGoodies, but I have moved over to using BBB because I find it's approach to be more intuitive. I can't stress the advantages of the declarative coding approach that binding allows for - it will truly revolutionize your code
AppFramework (or some flavor thereof) - JSR 296 is the place to start here. As I mentioned above, it has some problems - if you do use JSR296, I strongly, strongly recommend that you try to avoid using the singleton that is at the core of the framework (other than as a source for injection of the framework components that you actually need).
GlazedLists - if you are doing anything in the UI that involves lists, tables or trees, you should take a hard look at GlazedLists. It's an incredible project (not just for Swing apps, but it really shines in that arena)
Validation - JGoodies has a very good validation library. Learn it, use it, be one with it. Real time validation is an incredibly important part of a modern Swing app.
MigLayout - The Mig layout manager is the best around. I strongly advise against the temptation of using an IDE GUI builder - learn MigLayout (it will take a couple of hours, tops), and code things up by hand.
So those are the key, absolutely must-have libraries in my book.
Now some concepts:
A. Presentation Model - Martin Fowler has a lot of info on this design pattern. Long and short, it separates behavior from presentation at the GUI level. IF you are used to MVC, Presentation Model adds another layer of separation that is quite important to 'live' UIs. All of my views are backed by a corresponding presentation model. The end result is that the view code is really, really simple - focusing on two things: 1. Layout, and 2. Binding view components to the presentation model. That's it.
B. Views are NOT subclasses of JPanel. Instead, I follow the JGoodies inspired technique of treating the View as a builder that creates JPanels. The basic pattern is:
public class MyView{
private MyPresentationModel model;
private JButton okButton;
private JButton cancelButton;
...
public MyView(MyPresentationModel model){
this.model = model;
}
public JPanel buildView(){
initComponents(); // this method actually creates the okButton and cancelButton objects
bindComponentsToModel(); // this method binds those objects to the PresentationModel
JPanel p = new JPanel(new MigLayout());
p.add(...);
...
return p;
}
}
This approach, followed religiously, allows incredibly rapid development of UIs that are easy to maintain. Note that we can use a given View to construct multiple JPanels that are all backed by the same PresentationModel - changes in one panel generated by the view will immediately be visible in another panel generated by the same view.
C. Use Actions not event handlers. JSR 296 actually does a good job of making Actions easy to create and work with.
D. Do long running operations (even something that takes 100ms) off of the EDT. JSR 296 makes this fairly easy with it's Task support - but there are a number of gotchas in 296's Task system when it comes to exception handling. If you have property changes that in turn result in long running events, be sure you think carefully about which thread those changes are going to occur on. Using Tasks is a big change to how you do development, but it's a really important area for any real Swing application - take the time to learn about it.
E. Resource injection is important. Use it from the beginning (instead of telling yourself that you'll add it later) - if you find yourself calling setText() on a JLabel, it's time to sit back and call setName() instead, and add an entry to the resources file. JSR 296 makes this pretty easy to do if you are disciplined about it.
I think that's enough for now - this is a highly complex subject with lot's of nuance. I've spent the past 7 or 8 years banging my head into things that don't work - and I'm continually finding better ways to do even the things that work well for me.