Function call jumps to the wrong function.

Question

I am compiling a c++ static library in vs2008, and in the solution i also have a startup project that uses the lib, and that works fine.

But when using the lib in another solution i get an run-time check failure. "The value of ESP was not properly saved across a functioncall" Stepping through the code i noticed a function foo() jumping to bar() instead right before the crash. The functions in question are just regular functions and no function pointers.

Anyone has any clue what might be going on, and why it works when using the lib's from the same solution?

edit: the functions (methods) are part of a class, if that helps.

Answer 1

Make sure you're compiling in Debug mode and not Release mode. If you attempt to debug a program in Release mode, the data you get back from the debugger will be garbage due to optimizations.

Answer 2

This is most probably due to incompatible calling conventions, where the library and the caller have different ideas about stack layout.

Take a look at MSDN for more info.

Answer 3

I remember seeing such things when the struct member alignment (/Zp compiler switch) of the binaries was different. You could check that, too.

Setting it via #pragma pack rather than via project settings should be safer.

Answer 4

Forgive me for stating the bleeding obvious here, but... I've seen this sort of thing happen many times before when object (.o) and header (.h) files get out of sync. Especially with respect to virtual methods.

Consider: The object file is compiled with header:

class Foo { virtual void f(); };

But then the header gets changed to:

class Foo { virtual void g(); virtual void f(); };

And for the next object file, the compiler's assumptions about where f() is located in the class's vtable are incorrect.

Oftentimes simply recompiling the world (everything!) will help.

Answer 5

• Make sure that you haven't selected in your project an old version of a library, I.e. (as Adam mentioned) you have selected an older debug version instead of a current release version or vica versa.

• You might need to rebuild.

• Also watch out for conditional compilation where a macro might get #defined or #undef'd at some point (The other solution may have some macros or projected #defines). Some times it might be helpful to delete your .lib .obj, and precompiled header caches with rebuild.

• There is a remote possibility that something is messed up in your IDE or project configuration, where you might need to recreate you project from scratch.

• I am not too familiar with MS VC, can you add the library's project file from the other solution? For example, in Borland C++ Builder you can define project groups and build/make multiple projects, I always put the projects for the libraries I use in my program project group so everything gets up-to-date when I build.