Call stack corruption between boundaries

Question

This feels a lot like finding a needle in a hay stack but here goes.

I'm building a Windows Mobile 6.1 application. Specifically I'm trying to port over the OpenCV framework. Having successfully (doubtfully) compiled OpenCV for the ARM4I architecture, I'm trying it out in a simple hello world style application.

From my WinCE .EXE I'm calling a function stored in an OpenCV .DLL (cxcore200.dll). The simple call looks like this.

IplImage *src = cvCreateImage(cvSize(320,240), 8, 1);

Major problems arise when I step into cvCreateImage. The method signature is:

IplImage * cvCreateImage( CvSize size, int depth, int channels ){ ... }

So when I step into this function, depth and size params are equal to 320 and 240 respectively (not 8 and 1 as expected).

For reference, CvSize is declared as:

typedef struct
{
    int width;
    int height;
}
CvSize;

This is clearly some sort of call stack corruption that has to do with the fact that I'm crossing boundaries into possibly an incorrectly compiled DLL.

Both the DLL and EXE compile and link without errors. Has anybody ever seen anything like this? Any ideas on how to debug this?

Answer 1

I'm not familiar with your specific environment, but I'd suggest double-checking to make sure that you're using the correct calling convention when you call in to cvCreateImage. See this for reference material.

E.g., the default C++ calling convention in VC++ is "thiscall", but perhaps cvCreateImage expects __stdcall, __cdecl, or __fastcall.

Answer 2

When I had to debug this (15 years ago, Win16), I used the disassembly. Non-trivial to read, but it made unambiguous what precisely was happening.