I've been looking at WPF, but I've never really worked in it (except for 15 minutes, which prompted this question). I looked at this post but its really about the "Flash" of a WPF. So what is the difference between a Windows Forms application and a WPF application?
To answer your question, a WPF application is a Windows application. WPF is Microsoft's new framework (actually, it is a subsystem of the .NET framework 3.0) for writing rich Windows applications. It is meant as an eventual replacement for WinForms (although admittedly the adoption rate is much slower than MS hoped for).
WPF is a vector graphics based UI presentation layer where WinForms is not. Why is that important/interesting? By being vector based, it allows the presentation layer to smoothly scale UI elements to any size without distortion.
WPF is also a composable presentation system, which means that pretty much any UI element can be made up of any other UI element. This allows you to easily build up complex UI elements from simpler ones.
WPF is also fully databinding aware, which means that you can bind any property of a UI element to a .NET object (or a property/method of an object), a property of another UI element, or data. Yes, WinForms supports databinding but in a much more limited way.
Finally, WPF is "skinable" or "themeable", which means that as a developer you can use a list box because those are the behaviors you need but someone can "skin" it to look like something completely different.
Think of a list box of images. You want the content to actually be the image but you still want list box behaviors. This is completely trivial to do in WPF by simply using a listbox and changing the content presentation to contain an image rather than text.
A good way of looking at this might start with asking what exactly Winforms is.
Both Winforms and WPF are frameworks designed to make the UI layer of an application easier to code. The really old folks around here might be able to speak about how writing the windows version of "Hello, World" could take 4 pages or so of code. Also, rocks were a good deal softer then and we had to fight of giant lizards while we coded. The Winforms library and designer takes a lot of the common tasks and makes them easier to write.
WPF does the same thing, but with the awareness that those common tasks might now include much more visually interesting things, in addition to including a lot of things that Winforms did not necessarily consider to be part of the UI layer. The way WPF supports commanding, triggers, and databinding are all great parts of the framework, but the core reason for it is the same core reason Winforms had for existing in the first place.
WPFs improvement here is that, instead of giving you the option of either writing a completely custom control from scratch or forcing you to use a single set of controls with limited customization capabilities, you may now separate the function of a control from its appearance. The ability to describe how our controls look in XAML and keep that separate from how the controls work in code is very similar to the HTML/Code model that web programmers are used to working with.
A good WPF application follows the same model that a good Winforms application would; keeping as much stuff as possible out of the UI layer. The core logic of the application and the data layer should be the same, but there are now easier ways of making the visuals more impressive, which is likely why most of the information you've seen on it involves the flashier visual stuff. If you're looking to learn WPF, you can actually start by using it almost exactly as you would Winforms and then refactoring the other features in as you grasp them. For an excellent example of this, I highly recommend Scott Hanselman's series of blog posts on the development of BabySmash, which start here. It's a great walkthrough of the process, both in code and in thought.
Start here, there's dozens and dozens of videos: http://www.windowsclient.net/learn
In this case, the marketing schpiel is pretty good:
*"Windows Forms is a set of classes in the .NET Framework that enables rapid development of rich Windows client applications, with powerful, extensible libraries for user-interface controls and graphics. You can incorporate WPF in your Windows Forms applications through WPF-Windows Forms interoperability in the .NET 3.5 Framework."
*"WPF, a component of Microsoft .NET Framework 3.5, empowers you to build the next-generation of Windows user experiences. WPF supports UI, media, documents, hardware acceleration, vector graphics, scalability to different form factors, interactive data visualization, and superior content readability."**
With WPF you get WAY better graphics support, way more powerful and flexible databinding (if you're working with Data, use WPF.)