How to see the memory usage of NIO buffers?

Question

Is there a way to see the memory usage of NIO buffers? There is the non-heap memory section in JConsole, but I do not think this includes the NIO buffers?

Operating system is Linux (Ubuntu or CentOS) if that would matter.

regards,

Wim

Answer 1

Yes, from linux via the ps command. Example:

ps x -o command,rss | grep java | grep latest | cut -b 17-

The output will be in KB.

You may be interested in a question I had a while back regarding Java, RSS and NIO buffers: Why does the Sun JVM continue to consume ever more RSS memory even when the heap, etc sizes are stable?

Answer 2

I have used this to monitor the Virtual and RSS memory and the amount of native threads:

for((i=0;;++i)) { echo $i ` grep VmSize /proc/\21009/status | grep -o '[0-9]*'` ` grep VmRSS /proc/\21009/status | grep -o '[0-9]*'` ` grep Threads /proc/\21009/status | grep -o '[0-9]*'` ; sleep 1 || break; } > data

It creates a space separated file which you can easily import into your favorite spreadsheet tool.

You need to replace 21009 with the process id of the java process you want to monitor, of course.

This does not fully answer my own question as I wanted to be able to have a look at just what NIO stuff has allocated, not all the memory used by the JVM, although this little script already proved me useful enough to see that there are no memory leaks in a program I was investigating.

Answer 3

It is worth noting that direct memory buffers only use as much physical memory as you use (to the next page size, typically 4K). They use virtual memory when created but the OS is smart enough not to allocate pages of physical memory (resident) until you use the pages in the direct memory buffer. This means the size of physical memory used can be smaller than the amount of memory you allocate (virtual). The physical/resident memory is the one you should care about.

The benifit of this is you can allocate a few MB buffer even if you only expect to use a few KB and it will only use the amount you use and if for some reason you need more, your program won't blow up. i.e. you don't have to get the size right and can fairly safely over estimate the size allocated.