views:

913

answers:

3

Hi, I have a C program that stores some object in java store using JNI. (Before someone ask, using java store is a requirment here and I have to write a client in C which would be able to add and retrieve objects from this store).

I made the program and tried to add 100000 object of size 1KB. But after adding only 50000 objects I am getting 'out of memory' messages (please note that I am printing these 'out of memory' messages whenever I am unable to allocate a new string or byte array using NewStringUTF and NewByteArray functions). At that time my application is using only 80MB of memory. I dont why these methods are returning NULL. Is there something I am missing.

Furthermore, the memory keeps on increasing even though I am releasing byte array and string created for java.

Here is the source code.

    void create_jvm(void)
{
 JavaVMInitArgs vm_args;  
 JavaVMOption vm_options;

 vm_options.optionString = "-Djava.class.path=c:\\Store";
 vm_args.version = JNI_VERSION_1_4;
 vm_args.nOptions = 1;
 vm_args.options = &vm_options;
 vm_args.ignoreUnrecognized = 0;

 JNI_CreateJavaVM(&jvm, (void**)&env, &vm_args);

 if(env != null)
 {
  j_store = (*env)->FindClass(env, "com/store");
  if(j_store == null)
  {
   printf("unable to find class. class name: JStore");
  }  
 } 
}

void add(char* key, char* value, int length)
{
 jstring j_key = (*env)->NewStringUTF(env, key);
 jbyteArray j_value = (*env)->NewByteArray(env, length);

 (*env)->SetByteArrayRegion(env, j_value, 0, length, (jbyte *)value);
 ret = (*env)->CallStaticBooleanMethod(env, j_store, method_id, j_key, j_value);

 if(j_value != null)
 {
  (*env)->ReleaseByteArrayElements(env, j_value, (jbyte *)value, 0);
 }
 if(j_key != null)
 {
  (*env)->ReleaseStringUTFChars(env, j_key, key);
 }
}

The java side recieves the data in byte[] and stores it in a hashtable. The issue is that everytime the code runs the memory only adds up and is never released. I tried to add 1 MB object and debugged it.

The process memory increases by 1MB when I call NewByteArray. But when CallStaticBooleanMethod is called the process memory increase by 4MB. And the call to ReleaseByteArrayElements do not release any memory at all.

If I add another 1MB object after this, then process memory remains same when I call NewByteArray and it increase by 1MB when I call CallStaticBooleanMethod but remains the same when I try to release the byte array.

+2  A: 

The purpose of the functions ReleaseByteArrayElements and ReleaseStringUTFChars is not to delete the object but to unlock it after a pointer has been obtained with GetByteArrayElements or GetStringUTFChars. The two if statements should be removed.

Maurice Perry
I have removed these two statements but it didn't made any difference. The strange thing is that memory usage increases by 2MB to 4MB (when adding 1MB object) when I call the java method. Seems like the byte arrays are copying somewhere and its never garbage collected?
cornerback84
What is happening in the java method?
Maurice Perry
The code there is a little bit generic. Though I am passing in a byte[], the value is first serialized and then saved to a hashtable. You want me to post the java code too? (Serialization is part of requirement as the data may no always be in byte[] form and can be stored on disk or in another process)
cornerback84
Well that's the reason why twice the size of the twice the size of the object is allocated. Garbage collection only occurs when the heap size is reached. It seems to me that you need more space on the heap. The default limit is not that high. If you are getting out of memory error at 80Mb, it probably means that the default heap size is 64Mb. You can set the max. heap size with the -Xmx option.
Maurice Perry
+5  A: 

When you call New... functions, you create a "local reference" - reference to this object in local stack frame. This prevents Java VM from GC this object while you still need it. This is fine if you are implementing some native method - its local frame is created only for method call duration. But when you are creating object from a native java-attached thread, it becomes bound to this thread stack frame, which will be destroyed only with this thread.

So, when you done with an object you can call DeleteLocalRef() to tell you no longer need it. Or you can surround the whole add() function with pair of PushLocalFrame()/PopLocalFrame() to make a separate local frame for its duration.

Xeor
You mean whatever resource I get using new functions I can release for GC using DeleteLocalRef(). Something like this.jstring j_key = (*env)->NewStringUTF(env, key);jbyteArray j_value = (*env)->NewByteArray(env, length);.......(*env)->DeleteLocalRef(j_key);(*env)->DeleteLocalRef(j_value);
cornerback84
Yes, it tells jni that C code is released this resource - if java also doesn't have reference to it, it become free for GC
Xeor
What are the purpose of ReleaseStringUTFChars and other release functions? As far as I have read the documentation it states that it indicates that you no longer needs the resource, and should be freed. I think that was for the native functions (java to C) not the other way around then.
cornerback84
This seems to work. The memory is well under control. And remains around 80 - 100 MB.The solution posted by Maurice Perry (below) also worked, but in that case the memory goes to 500 MB before GC kick-in ( I have set Max heap size to 512MB). But in some case where I was using very large object (10MB), it sometimes did return null when I try to allocate byte array.But this solution keeps the memory in control.
cornerback84
A: 

Yes, I met the same issue. My java application call a C++ application by JNI, the C++ application will start a new thread, and call back a java method. In the new thread, many objects were created and the memory increase quickly, though I used DeleteLocalRef, PushLocalFrame and PopLocalFram. I found many objects that created by NewObject method cannot be released. It's strange .

Walter
New found, if there is a large string (printing/sending/passing) in the call back C++ method , the memory increase quickly, then overflow later
Walter