Memory randomization as application security enhancement?

Question

I recently came upon a Microsoft article that touted new "defensive enhancements" of Windows 7. Specifically:

• Address space layout randomization (ASLR)
• Heap randomization
• Stack randomization

The article went on to say that "...some of these defenses are in the core operating system, and the Microsoft Visual C++ compiler offers others" but didn't explain how these strategies would actually increase security.

Anyone know why memory randomization increases security, if at all? Do other platforms and compilers employ similar strategies?

Answer 1

It makes attacks like return to libc (or return to user-provided data buffer in the case of the latter two) much harder. And yes, it is available in Linux, BSD, and Mac OS. As you would expect, the details vary by OS. See Wikipedia for an introduction.

Answer 2

It increases security by making it hard to predict where something will be in memory. Quite a few buffer overflow exploits work by putting (for example) the address of a known routine on the stack, and then returning to it. It's much harder to do that without knowing the address of the relevant routine.

As far as I know, OpenBSD was about the first to do this, at least among the reasonably well-known OSes for PCs.

Answer 3

By randomizing the stack you make vanilla buffer overflow attacks like Aleph One's Smashing the Stack for Fun Profit impossible. The reason why is because the attack is relying on placeing a small ammount of executable code calld shellcode into a predictable location in memory. The function stack frame is corrupted and its return address overwritten with a value that the attacker chooses. When the corrupted function returns the the flow of execution moves to attacker's shellcode. Traditionally this memory address is so predictable that it would be identical on all machines that are running the same version of the software.

Despite advanced memory protection implemented on Windows 7 remote code execution is still possible. Recently at CanSecWest a machine running Windows 7 and IE 8 was hacked within seconds. Here is a technical description of a modern memory corruption attack utilizing a dangling pointer in conjunction with a heap overflow.