Why isn't there hardware support for memory management?

Question

Virtual memory is well-supported by modern hardware, but application-level memory allocation is still all implemented software, be it manual heap memory management a-la C and C++, or VM-level garbage collection.

Going further than the classic manual memory management vs garbage collection debate, why aren't there hardware-based primitives that can help build efficient memory management and/or garbage collection schemes in user-space (possibly exposed by, or built into the OS, and then various VMs).

Hardware is used to accelerate graphics operations, offload various parts of a network stack, and cryptographic algorithms and audio/video codecs are frequently implemented in hardware, why can't building blocks for higher-level memory management be? It seems so ubiquitous, yet I don't know any hardware-assisted implementations.

Given my lack of hardware knowledge it's a bit of a murky area to me, but I'm interested to hear

1. if there is such a thing at all (at least at the research stage), or
2. will or will it not give any benefit over conventional memory management, or alternatively
3. why it is not feasible to build such a thing in the hardware?

Answer 1

You could in theory implement a complete Java VM, including memory management, in hardware, and I believe there are some research projects that (try to) do that. But there are several good reasons not to implement stuff in hardware:

• hardware is fixed, you can't easily patch bugs or implement newer/better algorithms
• hardware is expensive, for complex operations such as garbage collection you'll need a lot of hardware, while a software implementation using existing hardware resources for it is much cheaper
• hardware resources take up space and consume (static) power, even while not in use, while unused software code does relatively little harm

In the end, for each feature you want, you have to make the trade-off between these costs, and the gain you have (faster or lower-powered execution).

For memory management, which are typically complex algorithms but that don't run all that often, the gains will be rather small (you may be able to speed up garbage collection by 10x, but if it only took 1% of execution time to begin with, why bother?) The cost, on the other hand, will be a much bigger chip where much of the area is wasted because it's inactive most of the time...

Answer 2

Yes, there were several CPUs which had memory management and GC built in. One was a custom version of the N320xx CPU that powered the Ceres workstation. It used 34bit memory (ie. 32bit data + 2bit extra).

There are several reasons why there is little hardware support for GC today:

1. You need a special mainboard -> expensive
2. You need a special CPU -> very expensive
3. You need special software that can use the extra features of the CPU and mainboard
4. There is still a lot of research going on how to make GC more efficient. This is a very active area, comparable to the time when we were drawing images by setting individual pixels. When we learn which parts can be standardized, it will make sense to build hardware for it.
5. It would waste memory for all programs which don't use this feature

[EDIT] The next generation of "general purpose CPUs" will probably come with a programmable area (FPGA) where you can define new "assembler op-codes". That would allow software to modify the CPU to its specific needs. The problem to solve here is to make the loading of the FPGA faster so its content can be switched with their processes.

This would allow to create hardware support for

Answer 3

In modern processors like pentium there are features which support virtual memory management. But the implementation of it has to be done by the OS, because there are so many possible algorithms how memory could be managed.

Which algorithm fits best depends on how the the memory is used. which kinds of applications are run on the computer? How long do they run? How many applications are run? And how are taskswitched organized.

You cannot hardwire this in hardware because of that. The operating system knows better how to efficiently manage memory, since it is made for a special tsype of computer (server vs. desktop OS), also it has a higher level view on the processes run on your computer.

Answer 4

In the embedded space, Ajile Systems Inc., http://www.ajile.com/ produce a series of JVM on a chip products which feature optional GC. They also offer a multiple JVM feature where java processes execute independantly on their own VM in a deterministic, time-sliced schedule with full memory protection.

They seem to offer three GC algorithms and an off mode. So not only a JVM on a chip more like an OS, of sorts, on a chip.

Answer 5

There are so many different algorithms and approaches to this problem that nobody has yet figured out any common primitives to them.

Answer 6

We had a lot of this hardware stuff in the 70th and 80th of the last millenium. All this Lisp machines were pretty good in trying to help memory management with indirect and double indirect access (required if your GC moves objects around). Some of us also remember the first days of the 80286 where people thought that segments could be used for better memory management and failed terrible on performance.

The current state of wisdom so far is that it is much better to optimize CPU's for general purpose usage instead of adding special features that are required only from time to time.

Modern garbage collectors already use some operating system features like the dirty markings of virtual pages to implement write barriers but other then this the algorithms are pretty simple, straightforward and high level. There isn't really special hardware required.

I just recently found an amazing result when using HP-UX. You can set the virtual page size to 256MB which will effectivly turn of the virtual memory. This gave a 120% performance increase on this CPU. TLB misses are really serious even more then cache misses. This makes me think about the good old MIPS architecture which stored a process id in the TLB so it did not require a complete TLB flush on each process switch.

There is still lot of room for memory management improvements that are more important then some high level Garbage collection features.