tags:

views:

183

answers:

1

Is there any way to debug what is going on after my .NET code calls a function in an unmanaged dll via dllimport?

I expose a dll function inside an unmanaged via dllimport. When I call a function as I step through the code, something happens and it never returns.

Is there anything I can do, maybe with debug view or anything to get any information? Or am I SOL?

+3  A: 

Well, for starters, make sure you have native code debugging enabled in the Debug tab of your project's properties.

If you have the source code for the DLL you are calling into, then you should be able to set it up so that you can step into the native DLL just like you can the managed code. The easiest way to do this is to add the source code for the native DLL in the same solution, and then link to the output of that project. You'll need to ensure that Visual Studio can access both the native DLL and that DLL's pdb while running your C# application.

If you do not have the source code for the DLL, then I'd suggest setting a breakpoint just before the call into the native method. Then, let the program continue and pause the program manually after a second or two. Make sure you're still looking at the same thread that called into the native DLL and then look at the call stack viewer. If you see a line that says "[External Code]", make sure that "Show External Code" is checked in the context menu. When that is checked, the call stack may be able to tell you what method is currently executing. The text will be in gray, indicating that the method is in an external DLL. If you are making Windows method calls, then installing the Windows Symbol Files first may allow the debugger to give you more information(http://www.microsoft.com/whdc/devtools/debugging/symbolpkg.mspx). You can also pause and continue the debugger several times to see what the call stack looks like at various points of execution. From this information, you may be able to get a sense of what is going wrong.

Also, try passing in different arguments, starting with the most trivial case you can and see if you get the same behavior. Finally, make absolutely sure that you have all your marshalling metadata correct.

Nimrand
Thorough answer thank you!
Matt