Is Private Bytes >> Working Set normal?

Question

OK, this may sound weird, but here goes.

There are 2 computers, A (Pentium D) and B (Quad Core) with almost the same amount of RAM running Windows XP.

If I run the same code on both computers, the allocated private bytes in A never goes down resulting in a crash later on. In B it looks like the private bytes is constantly deallocated and everything looks fine.

In both computers, the working set is deallocated and allocated similarly.

Could this be an issue with manifests or DLLs (system)? I'm clueless.

Also, I compiled the executable on A and ran it on B and it worked.

Note:

I observed the utilized memory with Process Explorer.

Question:

During execution (where we have several allocations and deallocations) is it normal for the number of private bytes to be much bigger (1.5 GB vs 70 MB) than the working set?

Answer 1

Memory leaks can be unpredictable and difficult to pin down. Since you have an IDE on the computer with the memory leak, and you are able to debug the process whose memory is increasing, traditional debugging methods on the bad process would be my only suggestion. You can't use valgrind since you aren't on *nix, but perhaps you should take a look at http://stackoverflow.com/questions/413477/is-there-a-good-valgrind-substitute-for-windows .

Answer 2

It could be a difference in fragmentation. For example, one might be using the small block heap and the other might not. The small block heap can help deal with fragmentation due to small allocations.

Answer 3

Virtual Address Space, Private Bytes and Working Set are 3 totally different aspects of your application.

• Private Bytes is the memory currently being used by your application.
• Working Set is the part of the memory that is currently loaded in RAM. The rest is swapped out in the page file.
• Virtual Address Space is the highest memory address that is ever being used by your application. Fragmentation may cause the Virtual Address Space to be much larger than the Private Bytes.

It is a typical misconception to think that the limit for Private Bytes is 2GB (for non-largeaddressaware 32-bit applications). 2GB is actually the limit for the Virtual Address Space. Fragmentation causes the limit for Private Bytes to be less. How much this is depends on the application. In my application I start to have problems around 1.7-1.8GB.

See http://shsc.info/WindowsMemoryManagement for a more thorough explanation.

Now this still doesn't explain the difference between your two computers. It's quite difficult to see what the actual cause of this difference is, but it might help looking at the 'aspects' in which they are different, and the first thing I see is the number of processors/cores. Does your application use multi-threading? Could there be a synchronization problem between your threads that only pops up on the quad-core system?

Answer 4

The fact that a memory leak (increasing private bytes) doesn't have an effect on working set is no surprise. The working set size is determined by the number of memory pages the application has touched recently. Private bytes is the amount of memory that a process has allocated (and not shared with other processes). If an application is forgetting to free objects that it is no longer using (a memory leak), then it's private bytes will not go down, but the working set will because it's not actively using that memory. See http://technet.microsoft.com/en-us/library/cc780836.aspx for details on the types of statistics for resources that Windows can track for a process.

You might want to look at the versions of the DLLs loaded by the application on each machine - it could be that a patch or service pack needs to be installed on the machine with the memory leak to fix the problem. Process Explorer can also show details of the DLLs loaded in a process.