Virtual Memory Usage from Java under Linux, too much memory used

Question

I have a problem with a java application running under linux.

When I launch the application, using the default maximum heap size (64mb), I see using the tops application that 240 MB of virtual Memory are allocated to the application. This creates some issues with some other software on the computer, which is relatively resource-limited.

The reserved virtual memory will not be used anyway, as far as I understand, because once we reach the heap limit an OutOfMemoryError is thrown. I ran the same application under windows and I see that the Virtual Memory size and the Heap size are similar.

Is there anyway that I can configure the Virtual Memory in use for a Java process under Linux?

EDIT: The problem is not the Heap. The problem is that if I set a Heap of 128M, for example, still linux allocates 210 MB of Virtual Memory, which is not needed, ever.

EDIT 2: Using ulimit -v allows limiting the amount of virtual memory. If the size set is below 204 MB, then the application won't run even though it doesn't need 204MB, only 64MB. So I want to understand why java requires so much virtual memory. Can this be changed?

EDIT 3: There are several other applications running in the system, which is embedded. And the system does have a virtual memory limit. (from comments, important detail)

Answer 1

No, you can't configure memory amount needed by VM. However, note that this is virtual memory, not resident, so it just stays there without harm if not actually used.

Alernatively, you can try some other JVM then Sun one, with smaller memory footprint, but I can't advise here.

Answer 2

Sun's java 1.4 has the following arguments to control memory size:

-Xmsn Specify the initial size, in bytes, of the memory allocation pool. This value must be a multiple of 1024 greater than 1MB. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. The default value is 2MB. Examples:

           -Xms6291456
           -Xms6144k
           -Xms6m

-Xmxn Specify the maximum size, in bytes, of the memory allocation pool. This value must a multiple of 1024 greater than 2MB. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. The default value is 64MB. Examples:

           -Xmx83886080
           -Xmx81920k
           -Xmx80m

http://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/java.html

Java 5 and 6 have some more. See http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

Answer 3

Just a thought, but you may check the influence of a ulimit -v option.

That is not an actual solution since it would limit address space available for all process, but that would allow you to check the behavior of your application with a limited virtual memory.

Answer 4

What number, exactly, are you using to say that virtual memory is "too high". Because, in general, that's a meaningless statement, and the virtual memory map contains a lot of things that don't actually consume active system resources.

For example, here's the "top" report for a Java process in my workstation (I think that it's NetBeans):

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
17463 kgregory  20   0  590m 214m  80m S  0.7 14.2  80:39.18 java

There are two numbers there: VIRT, which is the size of the virtual memory map, and RES, which is the amount of RAM consumed (there's also SHR, which is shared memory, but I'm going to focus on the first two).

OK, so if RES is big, in means that there's a lot of physical memory being consumed by your process, and that could interfere with other processes or cause a lot of paging.

The VIRT number, however, is much less meaningful. From the "top" documentation:

The  total  amount  of virtual memory used by the task.  It includes
all code, data and  shared  libraries  plus  pages  that  have  been
swapped out.

What this means is that VIRT includes stuff that's on-disk and rarely/never used. You can see exactly what this is by using the "pmap" command (showing only things loaded from the JDK directory; the full output is 617 lines):

08048000     60K r-x--  /usr/local/java/jdk-1.5/bin/java
08057000      8K rwx--  /usr/local/java/jdk-1.5/bin/java
909b0000   5632K r-xs-  /usr/local/java/jdk-1.5/jre/lib/i386/client/classes.jsa
911b0000   5932K rwx--  /usr/local/java/jdk-1.5/jre/lib/i386/client/classes.jsa
b1462000    384K r-x--  /usr/local/java/jdk-1.5/jre/lib/i386/libawt.so
b14c2000     24K rwx--  /usr/local/java/jdk-1.5/jre/lib/i386/libawt.so
b15ee000     68K r-x--  /usr/local/java/jdk-1.5/jre/lib/i386/libnet.so
b15ff000      4K rwx--  /usr/local/java/jdk-1.5/jre/lib/i386/libnet.so
b1681000   6880K r-xs-  /usr/local/java/jdk-1.5/lib/tools.jar

As you can see, there are many things in the virtual memory map: those that are marked "rwx--" are generally data spaces, while those that are marked "r-x--" are typically disk-resident, and affect your actual RSS/Swap in very indirect ways (tools.jar, for example, is a memory-mapped file; the JVM will access pieces of it in a random manner, and those will move into RSS but never into swap).

(edited to show a better extract from the pmap, with more explanation)

Answer 5

The amount of memory allocated for the Java process is pretty much on-par with what I would expect. I've had similar problems running Java on embedded/memory limited systems. Running any application with arbitrary VM limits or on systems that don't have adequate amounts of swap tend to break. It seems to be the nature of many modern apps that aren't design for use on resource-limited systems.

You have a few more options you can try and limit your JVM's memory footprint. This might reduce the virtual memory footprint:

-XX:ReservedCodeCacheSize=32m Reserved code cache size (in bytes) - maximum code cache size. [Solaris 64-bit, amd64, and -server x86: 48m; in 1.5.0_06 and earlier, Solaris 64-bit and and64: 1024m.]

-XX:MaxPermSize=64m Size of the Permanent Generation. [5.0 and newer: 64 bit VMs are scaled 30% larger; 1.4 amd64: 96m; 1.3.1 -client: 32m.]

Also, you also should set your -Xmx (max heap size) to a value as close as possible to the actual peak memory usage of your application. I believe the default behavior of the JVM is still to double the heap size each time it expands it up to the max. If you start with 32M heap and your app peaked to 65M, then the heap would end up growing 32M -> 64M -> 128M.

You might also try this to make the VM less aggressive about growing the heap:

-XX:MinHeapFreeRatio=40 Minimum percentage of heap free after GC to avoid expansion.

Also, from what I recall from experimenting with this a few years ago, the number of native libraries loaded had a huge impact on the minimum footprint. Loading java.net.Socket added more than 15M if I recall correctly (and I probably don't).

Answer 6

The Sun JVM requires a lot of memory for HotSpot and it maps in the runtime libraries in shared memory.

If memory is an issue consider using another JVM suitable for embedding. IBM has j9, and there is the Open Source "jamvm" which uses GNU classpath libraries. Also Sun has the Squeak JVM running on the SunSPOTS so there are alternatives.