I'm looking to learn how to build/program simple hardware and later move on to simple robotics. Where should I begin? What are the best sites to buy the hardware? Any recommendations on the type of hardware/language I should use to start?
Of course C/C++ for interfacing with the low-level bits of your robot, but also higher level-languages into which C/C++ can be easily embedded, like Python. Often it is the fastest way to start with a high-level language and use libraries that abstract away the difficult stuff (e.g. pyserial). Then, if you have your first behavior, you can dig deeper.
As a daily source of robotic inspiration, robots.net might be interesting.
Start with the Lego NXT. A full set of hardware + software for $250. It's a good starting point, and the applications people have done with it are very complete and serious. There's also a large community, so finding snippets of code, project ideas, and help in general is easy.
Take a look at Arduino. It's cheap (Arduino Pro currently available for $19.95 from SparkFun) and there's a free development environment which allows you to write in C/C++.
The following two books (although not specifically about robotics) are great as an introduction to hardware programming:
I got my start using the BASIC Stamp microcontroller (so named because it is programed in a dialect of BASIC and is about the size of a postage stamp) and robot kits from Parallax. They're a little pricey, but intended for beginners/hobbyists, extremely well-documented, and a lot of fun. I highly recommend them for a beginner. After I grew tired of forking out lots of money for underpowered microcontrollers (the on-chip BASIC interpreter sucks up most of the processing power), I graduated to PIC microcontrollers from Microchip. These are much more close-to-the-iron devices, and very powerful, despite the low cost ($4 for a small one). You'll need a programming setup, which can cost $50 as a hobbyist kit, or thousands for a production-quality system. They can be programmed in a wide range of languages, including C.
I would also recommend taking a look at Microsoft's robot studio. It has a decent simulation package for testing designs.