Recursive calls when trying to wrap and override the global operator new.

Question

I have not programmed C++ for a while and encountered a strange behavior when toying with overloaded global operators new and delete. The essence of the problem seems to be that a wrapper build around the default global new and residing in a separate source file nevertheless calls an operator new overloaded in another (and separately compiled so) source file.

Why is it so, i.e. which language rules / features am I violating / misusing?

Thanks in advance, the details are below.

Project structure:

.
..
main.cpp
mem_wrappers.h
mem_wrappers.cpp

Project files contents:

main.cpp

#include "./mem_wrappers.h"
#include <iostream>
#include <cstring>


void* operator new[] (size_t sz) throw (std::bad_alloc) {
  std::cout << "overloaded new()[]" << std::endl;
    return default_arr_new_wrapper(sz);
}


int main() {
    const unsigned num = 5;
    int * i_arr = new int [num];

    return 0;
}

mem_wrappers.h

#include <cstring>


void * default_arr_new_wrapper(size_t sz);

mem_wrappers.cpp

#include <new>
#include <cstring>
#include <iostream>


void * default_arr_new_wrapper(size_t sz) {
    std::cout << "default_arr_new wrapper()" << std::endl;
    return ::operator new[](sz);
}

Complied with g++ main.cpp mem_wrappers.cpp --ansi --pedantic -Wall when run gives endless operator new[] to default_arr_new_wrapper and vice versa calls resulting in the following output:

overloaded new()[]

default_arr_new_wrapper()

overloaded new()[]

default_arr_new_wrapper()

...

and, finally, in SO (the MS Visual Studio Express compiler behaves alike).

Compilers I use: gcc version 3.4.2 (mingw-special) and MS Visual Studio 2008 Version 9.0.30729.1 SP.

EDIT / UPDATE

If (as the first answers and comments suggest) the global operator new is effectively redefined for the whole executable just by overloading it [the operator] in a single compilation unit, then:

is it that merely linking with an object file whose source overloads the global operator new makes any application or library change its (well, lets call it so) memory allocation policy? Is thus this overload operator efficiently providing a hook for the language runtime (I mean what's with those linker symbols in the already-compiled-without-any-overloaded-new object files, is it that there can be only one new)?

P.S. I know that malloc and free would do, and already tried them before posting here (worked ok), but nevertheless, what's behind this behavior (and what if I was actually to wrap the default operator new anyway? :)) ?

Answer 1

I don't think there's much to add after Neil's comment but if you overload global operator new, you can't do it for just one translation unit, new will be overloaded for the entire executable or dll.

As you've noticed calling global operator new from within your own overloaded global operator new you will get endless recursion (well, until you run out of stack space at least ;)

If you want to toy around with this try using malloc and free inside your wrapper to get up and running.

If the operators were only local to one translation unit you would run into trouble if you passed an object which were allocated with one version of new to another translation unit and tried to delete it with another, non matching, operator delete.

The details of operator new and delete are covered in section 18.4 of the C++ standard.

Answer 2

To expand on Neil's comment, "global" new may not be given the static storage class specifier, and is therefore "global" - it is visible to the linker outside of the module (could be extern'd from any other module, for instance).

Invalid:

static void* operator new(size_t sz) {}

Valid:

//in main.cpp
void* operator new(size_t sz) {}

//in foo.cpp
extern void* operator new(size_t sz);

Depending on what you want to do, the best approach can be to use malloc/new in your definitions, or add a parameter to your override so the signature changes (you can then fall-back to the default global new without recursion).

Don't forget about throwing an exception in case of allocation failure, and implementing no-throw versions of your overloads.

Answer 3

If you read the wording in the standard, what you do to global operator new function when you define your own is called replacement (not overloading and not overriding). Most of the time when people talk about changing the global operator new functionality they use the term "overloading" and refer to new as an "operator". This often leads to confusion, when the resultant behavior does not agree with what is normally expected from operator overloading. This is apparently what happens in your case. You expect overloading behavior, but what you really get is replacement: once you defined one replacement version of global operator new function in some translation unit it works for the entire program. (And new is not really an operator.)

As a side note, replacement is one of those rare "unfair" language features, in a sense that it is not generally available to the user. You can't write "replaceable" functions in C++. You can't write anything that would behave the way default library implementations of global operator new and operator new behave. (Replacement is a language-level manifestation of the linker concept of a weak symbol).

Answer 4

To amplify what AndreyT has said:

When you write your own global operator new function you provide a definition that the linker can see. When it comes to link time, the linker will need to find a definition for new, just because its used a lot. It starts searching in the object files that you are asking it to link, and hey-presto there it is, so it can stop searching.

If you had not provided a definition, it would have carried on searching through any other object files of yours, then through libraries you provide, and finally through the run-time that comes with your compiler, which has to provide one or nothing would link.

Answer 5

Looking in the GNU C++ standard lib, you see decorations like this one, meaning "weak linking":

_GLIBCXX_WEAK_DEFINITION void *
operator new (std::size_t sz) throw (std::bad_alloc)