How to reliably measure available memory in Linux?

Question

Linux /proc/meminfo shows a number of memory usage statistics.

MemTotal:      4040732 kB
MemFree:         23160 kB
Buffers:        163340 kB
Cached:        3707080 kB
SwapCached:          0 kB
Active:        1129324 kB
Inactive:      2762912 kB

There is quite a bit of overlap between them. For example, as far as I understand, there can be active page cache (belongs to "cached" and "active") and inactive page cache ("inactive" + "cached").

What I want to do is to measure "free" memory, but in a way that it includes used pages that are likely to be dropped without a significant impact on overall system's performance.

At first, I was inclined to use "free" + "inactive", but Linux's "free" utility uses "free" + "cached" in its "buffer-adjusted" display, so I am curious what a better approach is.

When the kernel runs out of memory, what is the priority of pages to drop and what is the more appropriate metric to measure available memory?

Answer 1

I'd say it's hard to measure which pages, when dropped, would cause the system to have a "significant impact on overall system performance". Pages in use by the user processes are would be the (Total) - (Free + Cached + Paged). The 2nd term is all the memory the kernel could free if required. However, freeing memory pages being used for cache and pages would have a significant impact on overall system performance.

If I were going to use a heuristic, I'd say that you should take the value of "Inactive" which is "The total amount of buffer or page cache memory, in kilobytes, that are free and available. This is memory that has not been recently used and can be reclaimed for other purposes." If you find that you take that and the system continues to work fine, you could estimate some percentage of "Active" to take as a guess as well because the system might have used some pages recently but isn't going to use them again. You know more about the system than me. If the system is dedicated to whatever you're about to do, then the pages and cached files in Active that may be used soon would depend on whether the system was recently used for something else.

Answer 2

Since what “available memory” precisely means depends on your purpose, and your purpose is to avoid OOM situations:

Check out how Qt Extended (previously Qtopia) anticipates OOM situations.

There are two events:

• (MemFree + Buffers + Cached) / MemTotal < treshold (in /proc/meminfo)
• Major pagefaults > treshold (pgmajfault in /proc/vmstat I think)

The first is an early warning that memory is low, and triggers more frequent monitoring of pagefaults. The second signals trashing, which kills system performance and is a good hint the OOM killer will run.

Answer 3

At first I found your question easy, as in practice output from free in the columns '+ buffers/cache' is the one I use and it normally works.

But one of the situations in which it does not work is when you have heavy read to the same blocks. E.g. reading the same 1 gb_file over and over:

while true; do cat 1gb_file >/dev/null; done

If your system has > 1 GB cache then this will run fast. But if you start using some of that cache for something else it will smash the performance of the system.

So when you evaluate your solutions try the above and see if the solution takes that into account.