tags:

views:

11488

answers:

7

The question is not about the maximum heap size on a 32-bit OS, given that 32-bit OSes have a maximum addressable memory size of 4GB, and that the JVM's max heap size depends on how much contiguous free memory can be reserved.

I'm more interested in knowing the maximum (both theoretical and practically achievable) heap size for a 32-bit JVM running in a 64-bit OS. Basically, I'm looking at answers similar to the figures in a related question on SO.

As to why a 32-bit JVM is used instead of a 64-bit one, the reason is not technical but rather administrative/bureaucratic - it is probably too late to install a 64-bit JVM in the production environment.

+1  A: 

The limitations of a 32-bit JVM on a 64-bit OS will be exactly the same as the limitations of a 32-bit JVM on a 32-bit OS. After all, the 32-bit JVM will be running In a 32-bit virtual machine (in the virtualization sense) so it won't know that it's running on a 64-bit OS/machine.

The one advantage to running a 32-bit JVM on a 64-bit OS versus a 32-bit OS is that you can have more physical memory, and therefore will encounter swapping/paging less frequently. This advantage is only really fully realized when you have multiple processes, however.

Laurence Gonsalves
There may be a slight difference depending on the hardware and how it is virtualized. Some of the 4GB addressable space is generally used for memory mapped devices. The virtualization layer may or may not have the same memory footprint as the physical devices on the machine.
Eric J.
Not quite. There is more room for the JVM in a 64-bit machine since the 32-bit address space does not have to be shared with the operating system or hardware interfaces.
Thorbjørn Ravn Andersen
That's true, but all that means is that your 32-bit-virtual-machine might have slightly different overhead than a 32-bit-actual-machine (either worse, or better). Either way, you're running the JVM on a 32-bit-machine (real or virtual) and so you'll be subject to the standard 32-bit constraints. ie: an absolute ceiling of 4GB.
Laurence Gonsalves
Thorbjørn: the operating system and hardware interfaces still need to be mapped into the 32-bit VM. The precise amount of overheard may be different, but it'll still be there. If you can virtualize it out under an 64-bit OS what's to stop you from virtualizing it out under a 32-bit OS? This is *virtual* memory we're talking about.
Laurence Gonsalves
LG: I disagree with your original answer. The OS kernel and any HW and bus address space it maps will chew up lots of the address space, and while this isn't mapped into the user program, it does reduce the amount "left over" after the OS has set itself up. This is a considerable amount of a 4GB 32-bit space. Traditionally, this has meant that roughly 25%-75% of the 4GB is unavailable to user processes. :-) kernel hacker
DigitalRoss
@DR, Eric, Thorbjørn +1. That's precisely why a 32-bit OS cannot use all of the 4GB RAM available.
Vineet Reynolds
LG: regarding your comment. Certainly it is theoretically possible for the OS kernel and the user process to be in separate virtual spaces, and this has been true since the original i386. However, this is not how OS kernels are written, because it makes it so hard to move data back and forth from the kernel. It's interesting that some 16-bit systems, under severe address space pressure, actually could do this. I'm sure there are some obscure 32-bit ones, too, but AFAIK it's exceedingly rare.
DigitalRoss
Laurence Gonsalves, I encourage you to look up how much memory can be made directly available to a 32-bit process under a modern 64-bit operating system.
Thorbjørn Ravn Andersen
+6  A: 

You don't specify which OS.

Under Windows (for my application - a long running risk management application) we observed that we could go no further than 1280MB on Windows 32bit. I doubt that running a 32bit JVM under 64bit would make any difference.

We ported the app to Linux and we are running a 32bit JVM on 64bit hardware and have had a 2.2GB VM running pretty easily.

The biggest problem you may have is GC depending on what you are using memory for.

Fortyrunner
I would prefer to know the limitation for Solaris 10, but then that is only for my problem at hand. Would like to know for other OSes as well, for a rainy day :)
Vineet Reynolds
Not sure about Solaris. I would expect the VM size to be pretty big, my experience of Java on Solaris was from a few years ago. And being a Sun VM on a Sun OS on Sun hardware - things worked pretty well. I was also led to believe that there were fewer GC issues under Solaris than Linux/Windows.
Fortyrunner
Which Windows was this. I believe the server versions of Windows 32-bit can handle large amounts of memory much better.
Thorbjørn Ravn Andersen
Ah, finally a mention of the OS ;-) You have a 64-bit kernel installed?
Thorbjørn Ravn Andersen
It was Win2k server. A move to Win2k3 (things move slowly..) was too late and we switched to Linux instead.
Fortyrunner
Upvoted for your mention of the GC issue - very important for larger VM sizes.
shadit
+3  A: 

32-bit JVMs which expect to have a single large chunk of memory cannot use more than 4 Gb (since that is the 32 bit limit which must be for pointers). This includes Sun and - I'm pretty sure - also IBM implementations. I do not know if e.g. JRockit or others have a large memory option with their 32-bit implementations.

If you expect to be hitting this limit you should strongly consider starting a parallel track validating a 64-bit JVM for your production environment so you have that ready for when the 32-bit environment breaks down. Otherwise you will have to do that work under pressure, which is never nice.

Thorbjørn Ravn Andersen
No work pressure here. I know that a 64-bit JVM should have been installed in the first place. But it got me thinking about how a 32-bit JVM would work in an environment with more resources at its disposal.
Vineet Reynolds
Isn't it around 2GB becuase of the signednes? Or is that just the Sun JVM?
Sebastian Ganslandt
Pointers are not signed - it doesn't make sense to speak of negative memory locations.
Thorbjørn Ravn Andersen
No, it is not 2GB because of signedness. However, part of the 4GB address space is reserved for the OS kernel. On normal consumer versions of Windows, the limit is 2GB. On Linux and server versions of Windows (32-bit) the limit is 3GB per process.
Jesper
Jesper, please prepend with "Sebastian," when you answer a previous comment by Sebastian. I thought you responded to my comment.
Thorbjørn Ravn Andersen
+1  A: 

Should be a lot better

For a 32-bit JVM running on a 64-bit host, I imagine what's left over for the heap will be whatever unfragmented virtual space is available after the JVM, it's own DLL's, and any OS 32-bit compatibility stuff has been loaded. As a wild guess I would think 3GB should be possible, but how much better that is depends on how well you are doing in 32-bit-host-land.

Also, even if you could make a giant 3GB heap, you might not want to, as this will cause GC pauses to become potentially troublesome. Some people just run more JVM's to use the extra memory rather than one giant one. I imagine they are tuning the JVM's right now to work better with giant heaps.

It's a little hard to know exactly how much better you can do. I guess your 32-bit situation can be easily determined by experiment. It's certainly hard to predict abstractly, as a lot of things factor into it, particularly because the virtual space available on 32-bit hosts is rather constrained.. The heap does need to exist in contiguous virtual memory, so fragmentation of the address space for dll's and internal use of the address space by the OS kernel will determine the range of possible allocations.

The OS will be using some of the address space for mapping HW devices and it's own dynamic allocations. While this memory is not mapped into the java process address space, the OS kernel can't access it and your address space at the same time, so it will limit the size of any program's virtual space.

Loading DLL's depends on the implementation and the release of the JVM. Loading the OS kernel depends on a huge number of things, the release, the HW, how many things it has mapped so far since the last reboot, who knows...

In summary

I bet you get 1-2 GB in 32-bit-land, and about 3 in 64-bit, so an overall improvement of about 2x.

DigitalRoss
Unfortunately, I do not have a 64-bit environment at my disposal where I could experiment with the Xmx flag. The one that I know of has a humongous (32 * n)GB amount of RAM available, but out of bounds. That is why I wanted to know how a 32-bit JVM would work without all the constraints it normally faces in a 32-bit world.
Vineet Reynolds
Well, good question. I'm sure the basic answer is "it will work better".
DigitalRoss
Ok, I've edited my answer to focus a bit more on your actual question. :-)
DigitalRoss
≅ 3GB in 64-bit sounds just about right. Thorbjørn had already indicated why it theoretically cannot exceed 4GB. Too bad I cannot accept two answers.
Vineet Reynolds
No problem, you can upvote them both. :-)
DigitalRoss
If you have a _big_ box, you can experiment with a 64-bit Solaris in e.g. virtualbox (which has the best Solaris guest support).
Thorbjørn Ravn Andersen
A: 

I have tried setting the heap size upto 2200M on 32bit Linux machine and JVM worked fine. The JVM didnt start when I set it to 2300M.

Priya
A: 

From http://java.sun.com/performance/reference/whitepapers/tuning.html#section4.1.2 : "For a 32-bit process model, the maximum virtual address size of the process is typically 4 GB, though some operating systems limit this to 2 GB or 3 GB. The maximum heap size is typically -Xmx3800m (1600m) for 2 GB limits), though the actual limitation is application dependent. For 64-bit process models, the maximum is essentially unlimited. "

Found a pretty good answer here: http://stackoverflow.com/questions/171205/java-maximum-memory-on-windows-xp

djangofan
A: 

On Solaris the limit has been about 3.5 GB since Solaris 2.5. (about 10 years ago)

Peter Lawrey