Java -Xmx, Max memory on system

Question

My Java application runs another Java application, by running the process "java -jar j.jar". J.jar is known to use a LOT of memory depending on the dataset it is given, and often gets an OutOfMemoryError heap. So I want to use -Xmx on it, so that I can allocate as much memory as possible (or close to). I was thinking of getting the total memory on the system, then specifying 80-90% of that in -Xmx.

Is there any solution to my problem? And, how does my solution sound?

Edit: I cant reduce the memory consumption as the memory being used is by Java's built=in pack200 compression, which I am using to pack some JAR files.

Answer 1

Well, one thing I can tell you is don't let your app get close to filling up ram. Java apps don't swap gracefully at all. I think because of Garbage Collection, java constantly pulls its memory from swap.

I ran into a deadlock where I think the system was asking java for memory which would cause a GC and pull stuff out of the swapfile--at this point the system would just spin until I reset it.

This was with a LOT of ram and a LOT of swap space (for the time) and an older Java VM, so your mileage may vary.

Also, depending on how you are starting that other app, you may have to specify -Xms for your app instead of the other one. If you are giving it a full command, give it the -Xms, but if you are simply calling the main class in the jar, then your app needs the -Xms. (Oh, you specified, yeah you need to pass it into the "Java" command you are calling. )

Answer 2

Is there a reason you are using the OS to execute the program in the jar? If you don't need it to execute in a separate process from your application, you could just invoke the main method directly from your code, and start your application with whatever -Xmx you want.

Answer 3

Hello AmandeepGrewal:

There is a complete blog post on how to troubleshoot java applications with jconsole and other tools in the following blog. Keep in mind that the lack of control of memory use is most probably a memory leak, but it could be due to other reasons as well. Take a look to the post, try to replicate that scenario, and see if that solved your issue.

http://blog.readytocloud.com/2009/04/debug-java-application-in-cloud.html

Answer 4

If you haven't done so already, you need to run your program through a memory profiler. You might find that certain data structures aren't being disposed of, even though they're no longer being used.

JProfiler is pretty nifty, but you can get the same information using HPROF, which was introduced in Java 5: http://java.sun.com/developer/technicalArticles/Programming/HPROF.html

Keep in mind also that different platforms have different maximum heap sizes, based on architecture (32-bit vs 64-bit), OS, and even the JVM.

If you have a lot of values that may be reused (such as strings you're reading in from an XML file), you could get a huge decrease in memory requirements by pooling your objects.

Answer 5

Depending on your OS, this might work for getting the free and available memory size:

java.lang.management.OperatingSystemMXBean mxbean = java.lang.management.ManagementFactory.getOperatingSystemMXBean();
com.sun.management.OperatingSystemMXBean sunmxbean = (com.sun.management.OperatingSystemMXBean) mxbean;
long freeMemory = sunmxbean.getFreePhysicalMemorySize();
long availableMemory = sunmxbean.getTotalPhysicalMemorySize();

From there, you can figure out 80-90% and launch your jar with the max memory size you want.

I don't know that this works with all OS's (i.e. Windows), but it worked when I tested it with both OSX and Linux.

Answer 6

The limit for -XmX is -Xmx1500m on 32 bit windows. Shared libraries get in the way of a bigger heap. You'll need about 2Gb of RAM to do this.

On non-windows OSes you can go bigger, and 64Bit JVM's are capable of a LOT more.

Windows XP will not let you have more than 3Gb of RAM ( doesn't care if you have 4Gb physical, ever since XP SP3) Vista may be different YMMV.

I've tried -Xmx4000M on a 64 bit JVM on 64 bit Linux and it was fine. considering I had 6Gb of physical ram, it was not a big ask.

Your 80% idea is interesing, but my test systems run higher percentages than that without ill effect. (As long as you don't try doing anything else.)

And the other commenter is right, paging out your JVM's in-memory image is not quick. Later JVM's are better at doing this less messily ( but they have better garbage collectors too)

If you can't reduce yuor memeory consumprion: and I know how hardthat is then have lots of physical ram and allocate most of it.