Memory usage of FileChannel#map

views:

218

answers:

+2 Q:

Memory usage of FileChannel#map

Does FileChannel#map allocate all the memory needed for the resulting ByteBuffer immediately, or is it only allocated on-demand during reads from the buffer?

I just tried mapping all of a 500+ MB file in a trivial test program, and looked at the memory usage of the process. (Using both Runtime#totalMemory and eyeballing it in the OS X Activity Monitor for a groovysh process.) The memory usage never passed 30-ish MB.

So, could a Java implementation "hide" some of its memory usage in native calls? And if so, is there a way to find out how much that is on OS X?

+3 A:

Memory usage is never straightforward. The actual buffer used by FileChannel.map is not part of the Java heap. Indeed the memory might be shared with other processes. The file may not even be read from the disc until the pages are touched.

Tom Hawtin - tackline 2009-05-08 10:51:22

I'm not so sure about this, on some os (WinXP-32) it seems like some amount of buffer is consumed from the java heap. I have a test where consuming lots of java heap limits the quantity of fixed (max) size nio MemoryMappedBuffers I can create.

Justin 2010-08-02 21:23:18

@Justin I'm not sure how that would work. Obviously the `Buffer` object itself is in the Java heap, and there will be other small objects relating to the file mapping hanging about and perhaps some transient objects. Have you got a short test or figures?

Tom Hawtin - tackline 2010-08-02 22:40:08

Not specifically for determining numbers -- it was a side observation on another test, its all JVM dependent obviously. Try creating a big (4g) file and mapping it in (1024*1024*64 b) segments. The number of segments _seems_ to depend on how much JVM heap is _consumed_ by other objects in the jvm, not necessarily how much is available.

Justin 2010-08-03 14:06:15

@Justin The total size of the JVM heap will affect the amount you can map on a 32-bit system just because of virtual address space exhaustion.

Tom Hawtin - tackline 2010-08-03 14:53:23

Yes, I think that is related to what I am seeing. I _think_ that if you allocate allot of java objects it can break up the ammount of contiguous space which also matters. (see http://groups.google.com/group/hawtdb/browse_thread/thread/be81392389c586c5/5cd97256c9a09250)

Justin 2010-08-03 16:49:42

Yes. It's not part of the heap. Still, depending on the OS there will still be memory reserved. On OS X, because it's UNIX-like you should be able to use top or ps to check how much memory it needs.

kohlerm 2009-05-08 14:37:41

ansaurus

tags:

views:

answers:

Memory usage of FileChannel#map

related questions