An application's virtual bytes grow 2-times the private bytes.
does this indicate memory leak? bad application design?
OS is 32Bit
any thoughts are welcome. application is stream database.
+1
A:
Memory allocation has overhead to store management information about what was allocated. If you're allocating very small buffers the extra information can be a significant percentage of the total. That might be what you're seeing.
Jay
2010-07-27 16:33:43
+2
A:
An application's virtual bytes grow 2-times the private bytes.
If application allocates only heap, then to me it would be the sign that application allocates lots of memory but never actually touches it. For example:
void *p = malloc( 16u<<20 );
would eat up 16MB of virtual memory. But as long as application doesn't perform any actions with the memory block, OS wouldn't even attempt to map the virtual memory to the RAM. Simplest way to force the actual allocation of private memory is to memset() it:
void *p = malloc( 16u<<20 );
memset( p, 0, 16u<<20 );
does this indicate memory leak? bad application design?
Or both. Or neither.
The longer variant of the response: unknown, depends on what memory application allocates, what other resources application uses, OS, h/w platform, etc.
If unsure, use a memory leak analysis tools to investigate, e.g. valgrind. Read up SO for more information on memory leak analysis in C++.
Dummy00001
2010-07-27 16:37:40
That's only partially true. The C/C++ runtime writes values in parts of the allocated memory, mainly to remember the size of the allocated block (and probably some internal pointers as well). Remember that the C/C++ runtime is just a layer on top of the Windows heap manager, and the additional C/C++ memory management will probably cause the memory to be mapped anyway.
Patrick
2010-07-29 21:06:28
@Patrick: let's jsut say that my reply is as good as the question is. What you say is OK for a `std::list` or `std::map` type of structures - but using `std::vector`s one can already create rather large and sparse virtual memory space.
Dummy00001
2010-07-30 09:18:08
+1
A:
Fragmentation.
If you allocate the following chunks of memory:
- 16KB
- 8KB
- 16KB
and you then free the chunk of 8KB, your application will have 32 KB of private bytes, but 40 KB bytes of virtual memory, which is actually the highest virtual memory address that has ever been in use by your process (ignoring the other memory parts for sake of simplicity).
Consider (if possible) using another memory manager. Some alternatives are:
- The Windows Low-fragementation heap (see http://msdn.microsoft.com/en-us/library/aa366750%28VS.85%29.aspx for more info)
- The Doug-Lea open source memory manager
- Commercial alternatives like Hoard
A fourth alternative is to write your own memory manager. It's not that easy, but if done right, it can have quite some benefits. Especially for certain niche or special applications, writing your own memory manager be useful.
Patrick
2010-07-27 19:16:05