Keeping memory usage within available amount

Question

I'm writing a program (a theorem prover as it happens) whose memory requirement is "as much as possible, please"; that is, it can always do better by using more memory, for practical purposes without upper bound, so what it actually needs to do is use just as much memory as is available, no more and no less. I can figure out how to prioritize data to delete the lowest value stuff when memory runs short; the problem I'm trying to solve is how to tell when this is happening.

Ideally I would like a system call that returns "how much memory is left" or "are we out of memory yet?"; as far as I can tell, no such thing exists?

Of course, malloc can signal out of memory by returning 0 and new can call a handler; these aren't ideal signals, but would be better than nothing. A problem, however, is that I really want to know when physical memory is running out, so I can avoid going deep into swap and thereby making everything grind to a halt; I don't suppose there's any way to ask "are we having to swap yet?" or tell the operating system "don't swap on my account, just fail my requests if it comes to that"?

Another approach would be to find out how much RAM is in the machine, and monitor how much memory the program is using at the moment. As far as I know, there is generally no way to tell the former? I also get the impression there is no reliable way to tell the latter except by wrapping malloc/free with a bookkeeper function (which is then more problematic in C++).

Are there any approaches I'm missing?

The ideal would be a portable solution, but I suspect that's not going to happen. Failing that, a solution that works on Windows and another one that works on Unix would be nice. Failing that, I could get by with a solution that works on Windows and another one that works on Linux.

Answer 1

To find out how much system memory is still unused, under Linux you can parse the file /proc/meminfo and look for a line starting with "MemFree:". Under Windows you can use GlobalMemoryStatusEx http://msdn.microsoft.com/en-us/library/aa366589%28VS.85%29.aspx

Answer 2

The best solution I can think of might be to query how many page faults have occurred within, say, the last second. If there's a lot of swapping going on, you should probably release some memory, and if not, you can try allocating more memory.

On Windows, WMI can probably give you some statistics you can use.

But it's a tough problem, since there is no hard limit you can ask the OS for and then stay below. You can keep allocating memory far beyond the point where you've run out of physical memory, which just means you'll cripple your process with excessive swapping.

So the best you can really do is some kind of approximation.

Answer 3

I think the most useful and flexible way to use all the memory available is to let the user specify how much memory to use.

Let the user write it in a config file or through an interface, then create an allocator (or something similar) that will not provide more than this memory.

That way, you don't have to find statistics about the current computer as this will allways be biased by the fact that the OS could also run other programs as well. Don't even talk about the way the OS will manage cache, the differences between 32 and 64 bit making adress space limit your allocations etc.

In the end, human intelligence (assuming the user know about the context of use) is cheaper to implement when provided by the user.

Answer 4

You can keep allocating memory beyond the point where it is useful to do so - i.e. that which requires the OS to swap, or page out important things. The trouble is, it is not necessarily easy to tell where this is.

Also, if your task does any (significant) IO, you will need to have some left for the OS buffers.

I recommend just examining how much there is in the machine, then allocating an amount as a function of that (proportion, or leave some free etc).

Answer 5

Relying on malloc to return 0 when no memory is available might cause problems on Linux, because Linux overcommits memory allocations. malloc will usually return a valid pointer (unless the process is out of virtual address space), but accessing the memory it points to may trigger the "OOM killer", a mechanism that kills your process or another process on the system. The system administrator can tune this behavior.