views:

326

answers:

3

I am playing with an MSDN sample to do memory stress testing (see: http://msdn.microsoft.com/en-us/magazine/cc163613.aspx) and an extension of that tool that specifically eats physical memory (see http://www.donationcoder.com/Forums/bb/index.php?topic=14895.0;prev_next=next). I am obviously confused though on the differences between Virtual and Physical Memory. I thought each process has 2 GB of virtual memory (although I also read 1.5 GB because of "overhead". My understanding was that some/all/none of this virtual memory could be physical memory, and the amount of physical memory used by a process could change over time (memory could be swapped out to disc, etc.)I further thought that, in general, when you allocate memory, the operating system could use physical memory or virtual memory. From this, I conclude that dwAvailVirtual should always be equal to or greater than dwAvailPhys in the call GlobalMemoryStatus. However, I often (always?) see the opposite. What am I missing.

I apologize in advance if my question is not well formed. I'm still trying to get my head around the whole memory management system in Windows. Tutorials/Explanations/Book recs are most welcome!

Andrew

+1  A: 

That was only true in the olden days, back when RAM was expensive. The operating system maps pages of virtual memory to RAM as needed. If there isn't enough RAM to satisfy a program's request, it starts unmapping pages to make room. If such a page contains data instead of code, it gets written to the paging file. Whenever the program accesses that page again, it generates a paging fault, letting the operating system read the page back from disk.

If the machine has little RAM and lots of processes consuming virtual memory pages, that can cause a very unpleasant effect called "thrashing". The operating system is constantly accessing the disk and machine performance slows down to a crawl.

More RAM means less disk access. There's very little reason not to use 3 or 4 GB of RAM on a 32-bit operating system, it's cheap. Even if you do not get to use all 4 GB, not all of it will be addressable due hardware devices taking space on the address bus (video, mostly). But that won't change the size of the virtual memory accessible by user code, it is still 2 Gigabytes.

Windows Internals is a good book.

Hans Passant
Thanks. Will buy Windows Internals.
Dave
On a 32-bit machine, the 2 GB VA limit is per process. I bring this up because a common misconception is that it is across all processes. So user code can take advantage of more than 2 GB, so long as multiple processes are involved. There is also a /3GB Windows OS startup switch that makes this limit 3 gigabytes instead...
binarycoder
A: 

I don't know if this is your issue, but the MSDN page for the GlobalMemoryStatus function contains the following warning:

On computers with more than 4 GB of memory, the GlobalMemoryStatus function can return incorrect information, reporting a value of –1 to indicate an overflow. For this reason, applications should use the GlobalMemoryStatusEx function instead.

Additionally, that page says:

On Intel x86 computers with more than 2 GB and less than 4 GB of memory, the GlobalMemoryStatus function will always return 2 GB in the dwTotalPhys member of the MEMORYSTATUS structure. Similarly, if the total available memory is between 2 and 4 GB, the dwAvailPhys member of the MEMORYSTATUS structure will be rounded down to 2 GB. If the executable is linked using the /LARGEADDRESSAWARE linker option, then the GlobalMemoryStatus function will return the correct amount of physical memory in both members.

Since you're referring to members like dwAvailPhys instead of ullAvailPhys, it sounds like you're using a MEMORYSTATUS structure instead of a MEMORYSTATUSEX structure. I don't know the consequences of that on a 64-bit platform, but on a 32-bit platform that definitely could cause incorrect memory sizes to be reported.

Daniel Pryden
Thanks Daniel. I did see that on MSDN and have noted it. But, I don't think it affects me. The warning is saying that the dwAvailPhys membor could be under it's true value. It's truncated at 2GB. That's fine (for now). What I'm failing to understand is why the Physical Memory is ever more than the Virtual Memory. Again, it probably stems from my complete ignorance. It sure seems like dwAvailVirtual >= dwAvailPhys always, but this clearly is not the case. I have exactly 4 GB of memory (the 1stwarning is for >4 GB). I think I will try Ex just in case I'm getting bogus values.
Dave
A: 

The amount of virtual memory is limited by size of the address space - which is 4GB per process on a 32-bit system. And you have to subtract from this the size of regions reserved for system use and the amount of VM used already by your process (including all the libraries mapped to its address space).

On the other hand, the total amount of physical memory may be higher than the amount of virtual memory space the system has left free for your process to use (and these days it often is).

This means that if you have more than ~2GB or RAM, you can't use all your physical memory in one process (since there's not enough virtual memory space to map it to), but it can be used by many processes. Note that this limitation is removed in a 64-bit system.

slacker
Ah-hah! I think I was missing something basic and you (slacker) seem to be pointing it out. Is it the case that dwAvailVirtual refers to the available memory for that process whereas dwAvailPhys refers to the Available RAM for the entire system? Then it would make sense that often dwAvailPhys > dwAvailVirtual. The process could have used up almost all it's 2GB (or 4GB?), but there's still tons of physical RAM available (for other processes. Thanks,Andrew
Dave
As an aside, you say each process gets 4GB on a 32 bit system. But then you mention 2GB? What is the limit for each process? And what's the layman's explanation? How does 32 bit translate to 4GB?Thanks
Dave
It is exactly like that. Each process gets its own virtual memory space, while physical memory is (obviously) shared by the entire system. A 32-bit number can have 2^32 = 4G different values. If it is used as a byte-address, this means it can address 4G different bytes. That is the whole 32-bit address space. Now a part of it is reserved for kernel use, and kernel addresses can't be used in user code. On Win32, it is by default the upper 2GB of address space - leaving lower 2GB usable by the application.
slacker
Note that by using certain linker options you can tell Windows to reserve only 1GB for kernel, leaving 3GB to user. This mode is not the default, as it can break some older code relying on the "classic" address space division. So you must explicitly tell the system your app can handle it to have it turned on.
slacker
Also note that when your application is started, a bunch of system libraries are mapped into its address space together with its image. This means that from the beginning part of its address space is used. You can't count reliably on getting more than ~1.5G free for your own use (you'll probably get more, but the exact amount will vary depending on the system, so don't rely on it).
slacker