views:

782

answers:

10

Hi,

I'd like to write my own memory manager. The target language is C++ and the goal of the memory manager is mainly to help debugging. It should detect double frees, memory overwrite and so on. And of course - I'd like to learn about memory management.

Can someone give me a hint so resources where I can learn how to write such a memory manager?

Thank you for your help.

A: 

I have seen some samples use a C-preprocessor macro for malloc. Its a clever idea. I am sure you could write something like that.

Here is a looks like a good starting point.

http://stevehanov.ca/blog/index.php?id=10

Daniel A. White
A: 

Whats your platform? Just thinking whether valgrind or lint might be able to help you before you go and try to reimplement the kitchen sink?

Spence
sure - but I won't learn anything.
Tobias Langner
You won't learn anything in terms of coding technique by using Valgrind to spot your leaks? Surely if you are using a flawed way of destructing objects (or freeing depending on your platform) then you're going to find out very quickly.I do see value in implementing a heap etc. to see how it's done, but its a whole lot of effort in itself to do it right, and it leaves you with more things to debug when you have an issue.
Spence
If I'd do it for work - you're absolutly right. There is no reason to reimplement the wheel. But for the fun and for learning, just using valgrind does not give the insight and fun. But I appreciate your way of thinking Spence. The first thing you should do if you start programming is to check if someone already did your work. Most of the time they did it better then you could do.
Tobias Langner
nothin wrong with messing round with an API for fun. If I hadn't looked into SqlBulkCopy then I wouldn't have solved a showstopper at work. I thought if you were stuck at work though with a memory leak then I wanted to make sure you were aware of the pleasure (or pain...) that valgrind provides.
Spence
A: 

There is a great open source memory manager written in Delphi: fastMM4. It could be of value to have a look at it. It supports many of the features you want to implement and therefore might be a great showcase.

Ralph Rickenbach
A: 

You can implement most of it in terms of malloc and free - in fact a lot of C++ memory managers are implemented, even if they don't have to be.

You could start with a simple implementation that maintains a log of all allocations and deallocations, but forwards the regular allocations/deallocations to malloc and free. Obviously the simple implementation shouldn't use new/delete either...

So, to start you can

  • come up with the data structure that logs allocations and deallocations
  • Implement it mostly "C-style", although you can use placement new to ensure constructors are called
  • Implement global new and delete as wrappers that first log the access in the above data structure and then forward the call to malloc or free
Timo Geusch
+1  A: 

As @Spence said, this has already been done many times. But for the sake of learning it is quite interesting.

Might I suggest you take a look at ld's --wrap Here as it is useful

Salgar
This is a very good first step at intercepting new() and delete() calls. See Falaina's post on the complexities of detecting array overruns -- a different beast
Matt
+5  A: 

I think this is a very interesting project that you might learn a lot from. Here's a little bit of reading material on the subject of memory management. It goes over some of the basics of memory management, leads into a simple malloc implementation, then touches on a couple more advanced topics.

Inside memory management

Also, since you mention that you want to make a memory manager that is useful for debugging, you may want to consider reading this paper by the developers of Memcheck/Valgrind (A wonderful memory debugger for Linux). It details how they track all the the metadata (whether or not a particular byte is defined, intiaalized, etc.) in memchck. It's a bit detailed, but it's good reading on how to make a scalable and efficient dynamic memory checker.

How to Shadow Every Byte of Memory Used by a Program

Falaina
A: 

I think you could start with a smart pointer implementation which uses basic reference counting in the background. These are memory management basics and will get your feet wet. From there you can use your implementations to make a more advanced memory manager.

Polaris878
+1  A: 

Whatever you do, don't macro-override "new" like they did in MFC way back in the day. It interferes with placement new statements breaking the code.

MadKeithV
A: 

"Electric fence" is a basic starting point that might be helpful. Essentially it has custom implementations of malloc and free that provide the debugging.

However, AFAIK, it does not integrate into C++'s new/delete operators, though it would be not be too big of a stretch to provide custom global new/delete implementations that defer to the electric fence routines.

Michael Burr
+1  A: 

Dave Hanson's C Interfaces and Implementations presents first a standard memory manager and then a memory manager with a few debugging features. This would be a great starting point to learn from and extend.

Of course, if you actually want to diagnose memory problems in a running C or C++ program, you should use valgrind.

Norman Ramsey