I am starting to learn c# and wanted to create an actual app which one of the two would you start with?
Personally, I would learn WFP. We use a lot of Winforms, but we're in the process of migrating to WPF. I think that's a more future proof set of skills. WPF Virtual Labs are a good place to start.
The basics of either are much the same -- WPF offers a lot more features by way of declarative UI definition and skinning/theming, and a slightly different set of layout techniques. While you're at the "Hello world!" stages, it makes little difference -- certainly less than using either GTK# or the Swing implementation in VJSSupUILib would.
I would say learn both to have a stronger foundation of knowledge and be able to work with legacy code, but put more effort into WPF because it will eventually replace WinForms.
I recommend WPF - it is now very mature and well supported. Of course, winforms is still supproted and some new development is being done for it. However, WPF is significantly more sophisticated and powerful. Almost as importantly, the tool support for it is MUCH better than for WPF.
Here are some links for you.
My rule of thumb is to choose the most recent technology that doesn't require your users to go through extra effort. If you have to support Windows XP, then WinForms is the way to go as it doesn't require XP users to install .NET updates. If you don't have to worry about XP, then WPF is probably the ready to go.
I think that WPF should be used only when you need to show some animation, or maby another movement on the screen (graphs etc...), to replace GDI/GDI+ Last year was on MS conference, and there was one presentation about WPF in commercial apps, but guy who was talking about it did not convinced me to WPF as a technology for my future client. I saw more problem and issues than advantages.
But I'm learning it myself now, and I think it's a lot of fun and it's cool, so if you just want to learn it for yourself I wold choose WPF :)
Check out links from Foredecker.