Java OutOfMemoryError message changes when trying to create Arrays of different sizes

Question

In the question by DKSRathore How to simulate the Out Of memory : Requested array size exceeds VM limit some odd behavior was noted when creating an arrays.

When creating an array of size Integer.MAX_VALUE an exception with the error java.lang.OutOfMemoryError Requested array size exceeds VM limit was thrown.

However when an array was created with a size less than the max but still above the virtual machine memory limit the error message read java.lang.OutOfMemoryError: Java heap space.

Testing further I managed to narrow down where the error messages changes.

long[] l = new long[2147483645];

The exceptions message reads "Requested array size exceeds VM limit"

long[] l = new long[2147483644];

The exceptions message reads "Java heap space errors"

I increased my virtual machine memory and still produced the same result.

Has anyone any idea why this happens?

Some extra info:

Integer.MAX_VALUE = 2147483647

Edit: Here's the code I used to find the value, might be helpful:

int max = Integer.MAX_VALUE;
boolean done = false;
while (!done) {
    try {
        max--;
        // Throws an error
        long[] l = new long[max];
        // Exit if an error is no longer thrown
        done = true;
    } catch (OutOfMemoryError e) {
        if (!e.getMessage().contains("Requested array size exceeds VM limit")) {
            System.out.println("Message changes at " + max);
            done = true;
        }
    }
}

Answer 1

Maybe there is a global Java limit for the size of an array, whatever the available heap space is?

Answer 2

After playing around with this I think AndreaG is probably right. The language designers probably picked a relatively high number as the limit thinking "no one in their right mind would need an array this large."

Answer 3

Looking at the JDK 7 source code:

Check out the code here:

if (length > arrayOopDesc::max_array_length(T_ARRAY)) {
  THROW_OOP_0(Universe::out_of_memory_error_array_size());
}

Then you can see where the magic number comes from by looking at the definition of max_array_length here.

static int32_t max_array_length(BasicType type) {
  assert(type >= 0 && type < T_CONFLICT, "wrong type");
  assert(type2aelembytes[type] != 0, "wrong type");
  // We use max_jint, since object_size is internally represented by an 'int'
  // This gives us an upper bound of max_jint words for the size of the oop.
  int32_t max_words = (max_jint - header_size(type) - 2);
  int elembytes = (type == T_OBJECT) ? T_OBJECT_aelem_bytes : type2aelembytes[type];
  jlong len = ((jlong)max_words * HeapWordSize) / elembytes;
  return (len > max_jint) ? max_jint : (int32_t)len;
}

So the magic number is int max - the header size for an array - 2. I guess that means the header_size for this particular type is one, giving the magic number MAX_VALUE -3

Answer 4

As mentioned here,

3.1.3 Detail Message: Requested array size exceeds VM limit

The detail message Requested array size exceeds VM limit indicates that the application (or APIs used by that application) attempted to allocate an array that is larger than the heap size. For example, if an application attempts to allocate an array of 512MB but the maximum heap size is 256MB then OutOfMemoryError will be thrown with the reason Requested array size exceeds VM limit. In most cases the problem is either a configuration issue (heap size too small), or a bug that results in an application attempting to create a huge array, for example, when the number of elements in the array are computed using an algorithm that computes an incorrect size.

which leads to the conclusion that your heapsize just fits an array of size 2147483644, but not completely free resulting in the second messag. Allocating more than the heapsize results in the first message.