views:

237

answers:

6

I'm writing a memory tracking system and the only problem I've actually run into is that when the application exits, any static/global classes that didn't allocate in their constructor, but are deallocating in their deconstructor are deallocating after my memory tracking stuff has reported the allocated data as a leak.

As far as I can tell, the only way for me to properly solve this would be to either force the placement of the memory tracker's _atexit callback at the head of the stack (so that it is called last) or have it execute after the entire _atexit stack has been unwound. Is it actually possible to implement either of these solutions, or is there another solution that I have overlooked.

Edit: I'm working on/developing for Windows XP and compiling with VS2005.

A: 

I've read multiple times you can't guarantee the construction order of global variables (cite). I'd think it is pretty safe to infer from this that destructor execution order is also not guaranteed.

Therefore if your memory tracking object is global, you will almost certainly be unable any guarantees that your memory tracker object will get destructed last (or constructed first). If it's not destructed last, and other allocations are outstanding, then yes it will notice the leaks you mention.

Also, what platform is this _atexit function defined for?

Doug T.
You are guaranteed that destruction order mirrors construction.
Roger Pate
If that is true, then since construction order is not guaranteed, you cannot predict the mirrored destruction order. – Doug T. 0 secs ago
Doug T.
You can't force the order of construction *of globals in different translation units*. R.Pate's solution, which is to put all the globals in the same translation unit, does guarantee the order.
Steve Jessop
static Destruction does NOT mirror construction. Global destruction order does, but what about static function locals?
caspin
3.6.3: "Destructors for intialized objects of static storage duration (declared at block scope or at namespace scope) are called ... in the reverse order of the completion of their constructor". I simplify a bit, this is mixed in with the same rule applied to things which have initialization rather than construction.
Steve Jessop
@Caspin: __YES__ it does. The destructor of all static storage duration objects (including static function locals) are called in the reverse order of there constructors (guaranteed). If the constructor is not called (for static function locals) then the destructor is not called.
Martin York
@Doug: to be pedantic: construction order is not guaranteed __across__ compilation units.
Martin York
Grant Peters
Also, I should mention that the link I posted above solves the problem noted in the 'cite' by Doug T. in his answer
Grant Peters
A: 

Having the memory tracker's cleanup executed last is the best solution. The easiest way I've found to do that is to explicitly control all the relevant global variables' initialization order. (Some libraries hide their global state in fancy classes or otherwise, thinking they're following a pattern, but all they do is prevent this kind of flexibility.)

Example main.cpp:

#include "global_init.inc"
int main() {
  // do very little work; all initialization, main-specific stuff
  // then call your application's mainloop
}

Where the global-initialization file includes object definitions and #includes similar non-header files. Order the objects in this file in the order you want them constructed, and they'll be destructed in the reverse order. 18.3/8 in C++03 guarantees that destruction order mirrors construction: "Non-local objects with static storage duration are destroyed in the reverse order of the completion of their constructor." (That section is talking about exit(), but a return from main is the same, see 3.6.1/5.)

As a bonus, you're guaranteed that all globals (in that file) are initialized before entering main. (Something not guaranteed in the standard, but allowed if implementations choose.)

Roger Pate
Some libraries hide their global state in locally scoped variables with static storage duration. This also allows you to control the order, by deciding what order to call the functions that contain those static variables. Of course there are thread-safety issues.
Steve Jessop
@steve: I love that trick for fixing global construction order. However it has no effect on destruction order. local static variable destruction order is a complete free for all.
caspin
They're destroyed in reverse order of the completion of their constructors (3.6.3). This applies to all variables with static storage duration, regardless of their scope. So destruction order is a free-for-all if (and only if) construction order is. Granted, if not everybody uses the same scheme then namespace-scope globals can still cause things to be constructed before main() starts, in an unpredictable order if they're in different translation units, and you lose control.
Steve Jessop
Steve: thread safety is one reason why I prefer this solution: keep global state minimal and initialized at well-defined points---this does that succinctly and flexibly, using "before main" as that point.
Roger Pate
Also 18.3/8: "A local static object obj3 is destroyed at the same time it would be if a function calling the obj3 destructor were registered with atexit at the completion of the obj3 constructor". Not that the C++ standard would ever mandate implementation details. It just hints heavily ;-)
Steve Jessop
@R.Pate: sure, but if you're happy to require main.cpp to manage it, then thread safety can equally be achieved by having `main()` call a bunch of "initialise this" functions before creating any additional threads. The two systems are pretty much equivalent as far as that goes. Not sure what C++0x, or other C++ threading systems, say about what happens if you try to create a thread in a static constructor before `main()` is entered. And I'm not sure I want to know...
Steve Jessop
Indeed, essentially main() does handle it; it's just easier, simpler, and more manageable when coded as above. :)
Roger Pate
A: 

atexit is processed by the C/C++ runtime (CRT). It runs after main() has already returned. Probably the best way to do this is to replace the standard CRT with your own.

On Windows tlibc is probably a great place to start: http://www.codeproject.com/KB/library/tlibc.aspx

Look at the code sample for mainCRTStartup and just run your code after the call to _doexit(); but before ExitProcess.

Alternatively, you could just get notified when ExitProcess gets called. When ExitProcess gets called the following occurs (according to http://msdn.microsoft.com/en-us/library/ms682658%28VS.85%29.aspx):

  1. All of the threads in the process, except the calling thread, terminate their execution without receiving a DLL_THREAD_DETACH notification.
  2. The states of all of the threads terminated in step 1 become signaled.
  3. The entry-point functions of all loaded dynamic-link libraries (DLLs) are called with DLL_PROCESS_DETACH.
  4. After all attached DLLs have executed any process termination code, the ExitProcess function terminates the current process, including the calling thread.
  5. The state of the calling thread becomes signaled.
  6. All of the object handles opened by the process are closed.
  7. The termination status of the process changes from STILL_ACTIVE to the exit value of the process.
  8. The state of the process object becomes signaled, satisfying any threads that had been waiting for the process to terminate.

So, one method would be to create a DLL and have that DLL attach to the process. It will get notified when the process exits, which should be after atexit has been processed.

Obviously, this is all rather hackish, proceed carefully.

Tom
+1  A: 

This is dependent on the development platform. For example, Borland C++ has a #pragma which could be used for exactly this. (From Borland C++ 5.0, c. 1995)

#pragma startup function-name [priority]
#pragma exit    function-name [priority]
These two pragmas allow the program to specify function(s) that should be called either upon program startup (before the main function is called), or program exit (just before the program terminates through _exit). The specified function-name must be a previously declared function as:
void function-name(void);
The optional priority should be in the range 64 to 255, with highest priority at 0; default is 100. Functions with higher priorities are called first at startup and last at exit. Priorities from 0 to 63 are used by the C libraries, and should not be used by the user.

Perhaps your C compiler has a similar facility?

wallyk
I'm actually looking for this on VC++ and g++, great coincidence. I hope someone can elaborate.
GMan
I looked through the VS2008 docs, but didn't see anything. I even tried to use help on "#pragma" but it crashed the help facility. Sheesh!
wallyk
Thanks for trying. :) I know it's possible, I saw it somewhere once. If I find it, I'll let you know.
GMan
Please post if you do, I had a look through the MSDN documentation for this under the #pragma help, but it didn't appear to have anything like this.
Grant Peters
A: 

I've had this exact problem, also writing a memory tracker.

A few things:

Along with destruction, you also need to handle construction. Be prepared for malloc/new to be called BEFORE your memory tracker is constructed (assuming it is written as a class). So you need your class to know whether it has been constructed or destructed yet!

class MemTracker
{
    enum State
    {
      unconstructed = 0, // must be 0 !!!
      constructed,
      destructed
    };
    State state;

    MemTracker()
    {
       if (state == unconstructed)
       {
          // construct...
          state = constructed;
       }
    }
};

static MemTracker memTracker;  // all statics are zero-initted by linker

On every allocation that calls into your tracker, construct it!

MemTracker::malloc(...)
{
    // force call to constructor, which does nothing after first time
    new (this) MemTracker();
    ...
}

Strange, but true. Anyhow, onto destruction:

    ~MemTracker()
    {
        OutputLeaks(file);
        state = destructed;
    }

So, on destruction, output your results. Yet we know that there will be more calls. What to do? Well,...

   MemTracker::free(void * ptr)
   {
      do_tracking(ptr);

      if (state == destructed)
      {
          // we must getting called late
          // so re-output
          // Note that this might happen a lot...
          OutputLeaks(file); // again!
       }
   }

And lastly:

  • be careful with threading
  • be careful not to call malloc/free/new/delete inside your tracker, or be able to detect the recursion, etc :-)

EDIT:

  • and I forgot, if you put your tracker in a DLL, you will probably need to LoadLibrary() (or dlopen, etc) yourself to up your reference count, so that you don't get removed from memory prematurely. Because although your class can still be called after destruction, it can't if the code has been unloaded.
tony
Grant Peters
Hmm, of course you are right.I must be mixing up my memory with another case - I think in that case I did have to worry about threading, and had to be more careful with construction. It was a number of years ago.Either way, the "after destruction, output every time you are called" was definitely the technique I used for the exact problem mentioned. Maybe I should have stayed on topic.
tony
A: 

I've finally figured out how to do this under Windows/Visual Studio. Looking through the crt startup function again (specifically where it calls the initializers for globals), I noticed that it was simply running "function pointers" that were contained between certain segments. So with just a little bit of knowledge on how the linker works, I came up with this:

#include <iostream>
using std::cout;
using std::endl;

// Typedef for the function pointer
typedef void (*_PVFV)(void);

// Our various functions/classes that are going to log the application startup/exit
struct TestClass
{
    int m_instanceID;

    TestClass(int instanceID) : m_instanceID(instanceID) { cout << "  Creating TestClass: " << m_instanceID << endl; }
    ~TestClass() {cout << "  Destroying TestClass: " << m_instanceID << endl; }
};
static int InitInt(const char *ptr) { cout << "  Initializing Variable: " << ptr << endl; return 42; }
static void LastOnExitFunc() { puts("Called " __FUNCTION__ "();"); }
static void CInit() { puts("Called " __FUNCTION__ "();"); atexit(&LastOnExitFunc); }
static void CppInit() { puts("Called " __FUNCTION__ "();"); }

// our variables to be intialized
extern "C" { static int testCVar1 = InitInt("testCVar1"); }
static TestClass testClassInstance1(1);
static int testCppVar1 = InitInt("testCppVar1");

// Define where our segment names
#define SEGMENT_C_INIT      ".CRT$XIM"
#define SEGMENT_CPP_INIT    ".CRT$XCM"

// Build our various function tables and insert them into the correct segments.
#pragma data_seg(SEGMENT_C_INIT)
#pragma data_seg(SEGMENT_CPP_INIT)
#pragma data_seg() // Switch back to the default segment

// Call create our call function pointer arrays and place them in the segments created above
#define SEG_ALLOCATE(SEGMENT)   __declspec(allocate(SEGMENT))
SEG_ALLOCATE(SEGMENT_C_INIT) _PVFV c_init_funcs[] = { &CInit };
SEG_ALLOCATE(SEGMENT_CPP_INIT) _PVFV cpp_init_funcs[] = { &CppInit };


// Some more variables just to show that declaration order isn't affecting anything
extern "C" { static int testCVar2 = InitInt("testCVar2"); }
static TestClass testClassInstance2(2);
static int testCppVar2 = InitInt("testCppVar2");


// Main function which prints itself just so we can see where the app actually enters
void main()
{
    cout << "    Entered Main()!" << endl;
}

which outputs:

Called CInit();
Called CppInit();
  Initializing Variable: testCVar1
  Creating TestClass: 1
  Initializing Variable: testCppVar1
  Initializing Variable: testCVar2
  Creating TestClass: 2
  Initializing Variable: testCppVar2
    Entered Main()!
  Destroying TestClass: 2
  Destroying TestClass: 1
Called LastOnExitFunc();

This works due to the way MS have written their runtime library. Basically, they've setup the following variables in the data segments:

(although this info is copyright I believe this is fair use as it doesn't devalue the original and IS only here for reference)

extern _CRTALLOC(".CRT$XIA") _PIFV __xi_a[];
extern _CRTALLOC(".CRT$XIZ") _PIFV __xi_z[];    /* C initializers */
extern _CRTALLOC(".CRT$XCA") _PVFV __xc_a[];
extern _CRTALLOC(".CRT$XCZ") _PVFV __xc_z[];    /* C++ initializers */
extern _CRTALLOC(".CRT$XPA") _PVFV __xp_a[];
extern _CRTALLOC(".CRT$XPZ") _PVFV __xp_z[];    /* C pre-terminators */
extern _CRTALLOC(".CRT$XTA") _PVFV __xt_a[];
extern _CRTALLOC(".CRT$XTZ") _PVFV __xt_z[];    /* C terminators */

On initialization, the program simply iterates from '__xN_a' to '__xN_z' (where N is {i,c,p,t}) and calls any non null pointers it finds. If we just insert our own segment in between the segments '.CRT$XnA' and '.CRT$XnZ' (where, once again n is {I,C,P,T}), it will be called along with everything else that normally gets called.

The linker simply joins up the segments in alphabetical order. This makes it extremely simple to select when our functions should be called. If you have a look in defsects.inc (found under $(VS_DIR)\VC\crt\src\) you can see that MS have placed all the "user" initialization functions (that is, the ones that initialize globals in your code) in segments ending with 'U'. This means that we just need to place our initializers in a segment earlier than 'U' and they will be called before any other initializers.

You must be really careful not to use any functionality that isn't initialized until after your selected placement of the function pointers (frankly, I'd recommend you just use .CRT$XCT that way its only your code that hasn't been initialized. I'm not sure what will happen if you've linked with standard 'C' code, you may have to place it in the .CRT$XIT block in that case).

One thing I did discover was that the "pre-terminators" and "terminators" aren't actually stored in the executable if you link against the DLL versions of the runtime library. Due to this, you can't really use them as a general solution. Instead, the way I made it run my specific function as the last "user" function was to simply call atexit() within the 'C initializers', this way, no other function could have been added to the stack (which will be called in the reverse order to which functions are added and is how global/static deconstructors are all called).

Just one final (obvious) note, this is written with Microsoft's runtime library in mind. It may work similar on other platforms/compilers (hopefully you'll be able to get away with just changing the segment names to whatever they use, IF they use the same scheme) but don't count on it.

Grant Peters