Behringer got a nice package:
http://www.behringer.com/PODCASTUDIO-FIREWIRE/
It's not too expensive (ca. 199 € for the Firefire version, and ca. 100 € for the USB version which includes somewhat inferior hardware), but contains really good hardware and some software.
The hardest thing about the podcast is the content itself (speaking clearly and in the right speed, having a good pace throughout the show), but just like blogging this is something that can be learned through practice.
The rest is then just finding a Host, but usually a small shared hosting is good enough for the beginning and when you happen to become popular you can always look for a dedicated podcast hosting.
The things you have to check are the ways of offering standardized ways to retrieve them. For iTunes, Apple has Guidelines. For Blogs, there are Plugins and ways to create the RSS Feeds so that they can be accessed through Podcasting software. Have a look at Podpress or the source of the Stackoverflow Podcast RSS.