Largely because, to accomplish certain purposes, it's necessary. Just for example, C and C++ were originally used to write operating systems, including things like device drivers. To do that, they used (among other things) direct access to specific hardware locations that represented I/O devices. Preventing access to those locations would have prevented C from being used for its intended purpose (and C++ was specifically targeted at allowing all the same capabilities as C).
Another factor is a really basic decision between specifying a language and specifying a platform. To use the same examples, C and C++ are both based on a conscious decision to restrict the definition to the language, and leave the platform surrounding that language separate. Quite a few of the alternatives, with Java and .NET as a couple of the most obvious examples, specify entire platforms instead.
Both of these reflect basic differences in attitude about the design. One of the basic precepts of the design of C (largely retained in C++) was "trust the programmer". Though it was never stated quite so directly, the basic "sandbox" concept of Java was/is based on the idea that you should not trust the programmer.
As far as what languages do/don't have undefined behavior, that's the dirty little secret: for all practical purposes, all of them have undefined behavior. Some languages (again, C and C++ are prime examples) go to considerable effort to point out what behavior is undefined, while many others either try to claim it doesn't exist (e.g., Java) or mostly ignore many of the "dark corners" where it arises (e.g., Pascal, most .NET).
The ones that claim it doesn't exist generally produce the biggest problems. Java, for example, includes quite a few rules that try to guarantee consistent floating point results. In the process, they make it impossible to execute Java efficiently on quite a bit of hardware -- but floating point results still aren't really guaranteed to be consistent. Worse, the floating point model they mandate isn't exactly perfect so under some circumstances it prevents getting the best results you could (or at least makes you do a lot of extra work to get around what it mandates).
To their credit, Sun/Oracle has (finally) started to notice the problem, and is now working on a considerably different floating point model that should be an improvement. I'm not sure if this has been incorporated in Java yet, but I suspect that when/if it is, there will be a fairly substantial "rift" between code for the old model and code for the new model.