Given that you want a remote controlled vehicle capable of moving around, you will obviously need some kind of motors. I going to assume it will have differential drive locomotion since that's the simplest to build and very easy to control.
The simplest option is to get a pair of continuous hobby servos. They're cheap ($10 and up) and simple to drive through PWM (the duty cycle controls the velocity). If you want to step it up a bit you can get a pair of DC motors and encoders. You will need additional circuitry to drive them, but you can buy that ready made.
If you don't want it to be tethered you will need batteries and some kind of radio on board as well. If you're okay with not doing everything from scratch WiFi is the simplest solution in my opinion since you don't need any extra equipment on the computer you're using; it does tend to limit your "computer" and software options though (you'll want to run e.g. Linux on it).
You'll want an MCU/board that has:
- PWM, for driving your servos/motors
- A/D if you have analog sensors
- A bunch of digital I/Os for whatnot
- Built-in WiFi or a way to interface your WiFi card (USB, mini-PCI)
- SPI/I2C and UARTs for various peripherals
Having USB host signals has the added advantage that it's easy to hook up cheap cameras.
Something like the Roboard fits the bill (they also sell a mini-PCI wifi card). It packs plenty of computing power (and unlike many of the simpler boards, it has an FPU!), has plenty of connectivity options, including USB, and comes with a user space library to control the peripherals. (Disclaimer: I haven't tested it myself.)
It's not the cheapest hardware you can get, but it should have room to grow in if you want to try the more computing intensive parts of robotics (like SLAM, computer vision and motion planning). On the up side, you don't have to build a programmer/debugger for it.
If you'd rather start out simple, getting a cheaper board like the Arduino that someone already mentioned might be a better option. Using a simulator is also good to test out the software aspects before investing in hardware.
Where to start on the software side depends on what you're interested in.