The generally accepted answer is that you can't. However there is mounting evidence that this is not true based on the existence of projects that read in types of data that are not basic HTML types. Some projects that do this are the JavaScript version of ProtoBuf and Smokescreen.
Smokescreen is a flash interpreter written in JS so if it is not possible to get at the bytes directly how are these projects working around this? The source to Smokescreen can be found here. I have looked it over but with JS not being my primary language right now the solution eludes me.