RMI Thread terminating due to java.lang.OutOfMemoryError: Java heap space

Question

Hi All,

I am working on a comp based application. In this application there are n number of containers that communicate to each other via rmi services they provide to each other. At a certaion point One rmi thread conneccted to my container is lost the connection due to out of memory error ,but all other RMI thred connected to my container are working fine. The stack tarce of the error is here : Exception dispatching call to [655d565c:11f1d5dbae2:-7ffb, -3259564578052694518] in thread "RMI TCP Connection(21)-132.186.96.179" at Wed Jan 28 18:50:37 GMT+05:30 2009:

java.lang.OutOfMemoryError: Java heap space at java.lang.reflect.Array.newArray(Native Method) at java.lang.reflect.Array.newInstance(Unknown Source) at java.io.ObjectInputStream.readArray(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.readObject(Unknown Source) at sun.rmi.server.UnicastRef.unmarshalValue(Unknown Source) at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source) at sun.rmi.transport.Transport$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Unknown Source) at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

to view this exception i had to actiave rmi speific logging & this problem was occured because each RMI call of this terminated Thread adds some data to heap of My continer. and at certain point it this size exedes.

My question to you guys is that if it is out of memory in heap size of my container why other threads are working? pls let me know if you have any idea.

Answer 1

The failed call is

java.lang.reflect.Array.newArray(Native Method)

which means that the failing RMI thread was trying to allocate an array. Unfortunately, it doesn't tell us how big an array it was trying to allocate. It it was trying to allocate a huge array, and failed, then this would not hurt any other thread. Is there anything different about the failing request that it would have to allocate much more memory than the other requests that are being made?

To elaborate ... let's say that for some reason this one request is trying to allocate a 500 meg array (and there isn't enough memory on the heap). Well, that allocation request will fail. But as long as the heap still has enough memory for normal allocation requests, the other threads won't have a problem creating new objects.

Answer 2

Simple answer: because other operations don't have the memory requirements of the operation that fails.

Longer answer starts with looking at the stack trace:

java.lang.OutOfMemoryError: Java heap space at java.lang.reflect.Array.newArray(Native Method) at
java.lang.reflect.Array.newInstance(Unknown Source) at java.io.ObjectInputStream.readArray(Unknown Source) at
java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown
...
sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) at

So, the thread that's failing is trying to create a new array, in order to process an incoming request. If that array is particularly large, then the request handler may be unable to allocate it, while other requests that use smaller amounts of data will process without a problem.

So, the first question is: how big is that array? You should put some logging on the client side to determine this. And if a particular client is trying to send an exceptionally large array, you might need to reduce its size.

Second question: have you given the server JVM enough memory to handle the expected load? Jave is one of the few environments that still require you to tell it how much memory a program needs, and the default is quite small (64M I believe). Look into the -ms and -mx parameters to learn how to increase it.