It may be possible to turn on and off power to the device by doing whatever happens when you "safely remove hardware" a USB device, but I'm not 100% sure this turns off the power, it may just send a signal to the device tell it it's disconnected, which makes it turn itself off.
The best bet for what you're trying to do is to build a USB device that actually communicates with the computer (possibly via a custom driver, but you can probably also use a standard driver like a HID device). You would have the computer send "on" or "off" (or anything else you can dream up) to your device, which would interpret the signals as you see fit (probably by turning on and off that LED you were talking about). This isn't as hard as it sounds. You can buy microcontrollers with USB interfaces on board, such as the Arduino, or some of the high-end PICs. Many of the manufacturers of these sorts of microcontrollers have documents available explaining how to do USB interfacing with them.