C++ STL in VS2008: std::ostringstream throws std::bad_alloc after heavy assign/clear usage

Question

Hi all,

I have come across a situation (on Win32) where the std::ostringstream object continues to consume process memory, even when it is ostensibly cleared out after a series of append-type operations. Please take a look at this C++ fragment:

int main(void)
{
    std::ostringstream cOutputLogStream;

    // Random long string
    std::string sTest = "jkspoiauyeraspfoiusdfsdfekgpweojkgpwoekpokgkpgeopoegwj";

    std::string sEmpty = "";

    int n = 0;
    int looper = 0;

    while (n++ < 100000)
    {
        while (looper++ < 45)
        {
            cOutputLogStream  <<  s;
        }

        cOutputLogStream.str(sEmpty);
        cOutputLogStream.clear();

        // This should give the heap manager a chance to consolidate
        // fragmented memory blocks
        Sleep(1);
    }
}

During execution of the inner while() loop, observation of the process's memory usage in Task Manager shows a continuous upward ramp, which eventually levels off. However, this levelling-off happens at the same time as when the error std::bad_alloc is repeatedly thrown. This would suggest that heap memory has either been exhausted, or the requested size of block is not available in a contiguous space.

Has anyone else experienced this leaking phenomenon with ostringstream objects, and what other alternative objects are available instead of this flaky one?

Many thanks!

Answer 1

I don't see how this code could repro the problem. After looper increments to 45, it should be about done consuming any memory.

The generic diagnostic is that a program very rarely manages to consume all possible available virtual memory. It will die first on finding a chunk of contiguous bytes inside the heap that's large enough to store the string stream buffer. It is called address space fragmentation, there isn't anything you can do about it. Your Sleep() call will certainly not do anything useful, consolidating allocated heap blocks requires a garbage collector.

Another pretty standard trap is using TaskMgr.exe to diagnose memory usage. It normally displays the working set, the amount of virtual memory that's mapped to RAM. That's usually only a fraction of the amount of virtual memory your program consumes and cannot give a true measure of how much virtual memory your program has consumed. Or tell you anything about address space fragmentation for that matter.

The SysInternals' VMMap utility can show you how your program is using virtual memory.