std::vector reserve method fails to allocate enough memory

Question

Hi,

I have a buffer class in my C++ application as follows:

class Buffer
{
    public:
    Buffer(size_t res): _rpos(0), _wpos(0)
    {
     _storage.reserve(res);
    }

    protected:
    size_t _rpos, _wpos;
    std::vector<uint8> _storage;
}

Sometimes using the constructor fails because its unable to allocate the required memory space. For example, once, calling the constructor with res = 37 caused a segfault with the following stack trace that i got from its core dump:

#0  0x00007f916a176ed5 in raise () from /lib/libc.so.6
No symbol table info available.
#1  0x00007f916a1783f3 in abort () from /lib/libc.so.6
No symbol table info available.
#2  0x00007f916a1b33a8 in ?? () from /lib/libc.so.6
No symbol table info available.
#3  0x00007f916a1b8948 in ?? () from /lib/libc.so.6
No symbol table info available.
#4  0x00007f916a1bb17c in ?? () from /lib/libc.so.6
No symbol table info available.
#5  0x00007f916a1bca78 in malloc () from /lib/libc.so.6
No symbol table info available.
#6  0x00007f916ac0c16d in operator new (sz=37)
    at ../../.././libstdc++-v3/libsupc++/new_op.cc:52
        p = <value optimized out>
#7  0x00000000004e3d11 in std::vector<unsigned char, std::allocator<unsigned char> >::reserve (this=0x7f911bc49cc0, __n=31077)
    at /usr/local/lib/gcc/x86_64-unknown-linux-gnu/4.4.2/../../../../include/c++/4.4.2/ext/new_allocator.h:89
        __old_size = 0
        __tmp = <value optimized out>

I've compiled this application using GCC 4.4.2 as a 64 bit application and I'm using it in Debian 5 x64.

Any help is much appreciated. Thanks

Answer 1

Because the segfault is in malloc, most likely some other code has trashed the heap (i.e. written to parts of memory they do not own and are being used by the heap manager).

I recommend using Valgrind to find what code is trashing the heap.

Answer 2

If you can't use Valgrind to find out where your memory is corrupted because of the heavy load it implies, you can still test with lighter solutions.

For server application where Valgrind was not applicable (because the platform was on Solaris 8), I had pretty good result with mpatrol ( http://mpatrol.sf.net ) but especially dmalloc ( http://dmalloc.com ).

To some extend, you can use them without recompiling (just relinking for dmalloc, library preloading for mpatrol). They'll replace the memory primitives to perform extra checks on the memory use (bad argument to those primitives, reading off-by-one, heap corruption, ...) Some of those checks will be triggered exactly when the problem occurs while others will be triggered a bit later than the actual bad code. By tuning which checks are enabled, and when applicable the check frequency, you can run at almost full speed while performing basic checks.

I recommend recompiling with dmalloc to get so called 'FUNC_CHECK', for me, this added a lot of accuracy in bug spotting with a negligible performance cost.