I have a piece of an application that is written in C, it spawns a JVM and uses JNI to interact with a Java application. My memory footprint via Process Explorer gets upto 1GB and runs out of memory. Now as far as I know it should be able to get upto 2GB. One thing I believe is that the memory the JVM is using isn't visible in the Process Explorer. My xmx is set to 256, I added some statements to watch the java side memory and it is peaking at 256 and GC is doing its job and it is all good on that side. So my question is, where is the other 700+ MB being consumed? Anyone out there a Java/JNI/C Memory expert?
A:
Write a C test harness and use valgrind/alleyoop to check for leakage in your C code, and similarly use the java jvisualvm tool.
Dan
2008-09-19 18:48:57
A:
The C and JNI code can allocate memory as well (malloc/free/new/etc), which is outside of the VM's 256m. The xMX only restricts what the VM will allocate itself. Depending on what you're allocating in the C code, and what other things are loaded in memory you may or may not be able to get up to 2GB.
John Gardner
2008-09-19 18:53:28
A:
If you say that it's the Windows process that runs out of memory as opposed to the JVM, then my initial guess is that you probably invoke some (your own) native methods from the JVM and those native methods leak memory. So, I concur with @John Gardner here.
Alexander
2008-09-19 19:05:04
Well, as far as I can tell the C side is getting 1GB, the JVM is using 256MB so 1024+256 = 1280. I would expect to be able to use 768 more. Where is that 768MB being consumed, if it isn't being used on the C side and it isn't used by the JVM.
drye
2008-09-19 19:14:26
+1
A:
There could be a leak in the JNI code.
Remember to use (*jni)->DeleteLocalRef() for any object references you get once you are done with them. If you use any native C buffers to create new Java objects, make sure you free them off once the object is created. Check the JNI Specification for further guidelines.
Depending on the VM you are using you might be able to turn on JNI checking. For example, on the IBM JDK you can specify "-Xcheck:jni".
fd
2008-09-19 19:25:21
+1
A:
Try a test app in C that doesn't spawn the JVM but instead tries to allocate more and more memory. See whether the test app can reach the 2 GB barrier.
Alexander
2008-09-19 20:00:48
I did give you a vote on this, as I had done this test. I also made it allocate 1.4GB before starting the JVM and when trying to start the JVM it was unable to allocate enough memory for the HEAP object. So I have come to the conclusion that it is the HEAP that is taking up the unseen memory.
drye
2008-09-19 20:35:03
Well then, check how much memory is used for memory-mapping all the dynamic libraries used by the JVM and your C app. May be it's this memory that's to blame.
Alexander
2008-09-19 20:40:28
A:
Well thanks to all of your help especially @alexander I have discovered that all the extra memory that isn't visible via Process Explorer is being used by the Java Heap. In fact via other tests that I have run the JVM's memory consumption is included in what I see from the Process Explorer. So the heap is taking large amounts of memory, I will have to do some more research about that and maybe ask a separate question.
drye
2008-09-19 20:38:27