Max amount of memory per java process in windows?

Question

This is a pretty simple question:

What is the maximum heap size that you can allocate on 32-bit Windows for a Java process using -Xmx?

I'm asking because I want to use the ETOPO1 data in OpenMap and the raw binary float file is about 910Mb.

Answer 1

As noted in the question mentioned in the comment, there is a practical limit circa 1200mb.

However the situation you're describing has more depth to it than sheer memory size.

When you read a 910MB binary data and build a network objects off of it (as opposed to just maintaining the data as an array of bytes), you end up consuming much more memory than 910MB. A reasonable estimate would be that the in-memory representation will consume twice as much memory - that's because (1) each object contains an additional pointer (to the class of the object); and (2) there's a lot bookkeeping data. For instance if you use a HashMap to manage your objects then in addition to each object you also allocate a Map.Entry object which can easily consume 16 or 20 bytes (impl. dependent).

On the other hand, there's still hope: do you really need to maintain all 910MB in memory? Can't you just build something that reads the data in lazy manner? Combined with WeakReferences I think you can pull this off.

Answer 2

In 32 bit Windows, by default, every application can use up to 2GB virtual address space. I guess this makes -Xmx2048M. However, if you have more ram istalled, you can increase the virtual address space up to 3GB by using boot time parameters.

In boot.ini, you can create a new boot options like this:

[boot loader]
timeout=5
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional - magyar" /noexecute=optin /fastdetect /usepmtimer
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional - magyar 3GB" /noexecute=optin /fastdetect /usepmtimer /3GB /USERVA=2800

Here by adjusting the /USERVA=2800 parameter, you can tune your machine. But be aware that some configurations don't like high values in this parameter - expect crashes.

Answer 3

For a large file I suggest you use a memory mapped file. This doesn't use heap space (or very little) so maximum heap size shouldn't be a problem in this case.

Answer 4

We have recently ported from Windows to Linux (because of VM size issues).

I have heard of lots of numbers thrown around in the past for Windows VM size (1200, 1400, 1600, 1800). On our Windows Servers (2003), in our environment, with our applications, ... I have never successfully used more than 1280MB. Beyond that our application started exhibiting GC and OOM issues.

Everytime I got a new VM version I tried changing the number and it never varied.

You have a 900MB file now, what if the file increases to 1300MB? What will you do?

You have a number of options

1. Port to Linux/Solaris. This just needs hardware/software and what is often a simple porting exercise.
2. Use 64bit Windows. This may not be free of GC issues though - I have heard of different tales with 64bit vms.
3. Redesign the app to process the file differently, Can you split the file logically in some way, can you read the file in chunks and process it differently etc?

Other people using OpenMap must have encountered this issue. Can you tap into their knowledge and not re-invent any wheels?