PIC Microcontrollers from Microchip are a great way to get into embedded programming, because the hardware is so simple. All you need in terms of circuitry is a ceramic resonator, a capacitor for the power supply, and a resistor for the reset. These are all very cheap.
Get the 3-pin kind of ceramic resonator - 2 pins go directly to the PIC, the centre pin goes to ground. Put an electrolytic capacitor of about 0.1 or 1uF between +5v and ground, and a 10k resistor between the MCLR pin (normally pin 1) and +5v. That's all the hardware you need! You really can't get much simpler. DON'T think you can skip the capacitor though, even if you're just turning a single LED on and off. Your CPU will reboot at random times and you'll have nightmares trying to figure out why.
You then connect it to your PIC programmer (there are different kinds available, from cheap Chinese knock-offs to expensive in-circuit emulators), flash the chip, and you're done. Chips are cheap, and the flash kinds can be reprogrammed many times. (Be sure to get the FLASH ones, not the one-time-programmable (OTP) ones! The flash ones have a part number like xxFxxxx, whereas the OTP ones are xxCxxxx.)
All these components are cheap and easily available from mail-order suppliers. In the UK, I recommend Rapid Electronics and Farnell.
There is loads of documentation and samples for PICs available directly from Microchip's website. They also give away a free IDE, assembler, and a trial C compiler.
The microcontrollers have lots of on-board peripherals, including analog->digital converters, RS-232, timers, interrupts, I2C, CAN, and even USB. More than enough to get you started in the world of embedded programming.