Debugging of native code

views:

answers:

+1 Q:

Debugging of native code

I have a C# Service that is calling a C DLL that was originally written in VC6.

There is a bug in the DLL which I am trying to inspect.

After having a nightmare trying to get debug to work I eventually added the dll to the VS2005 solution containing the C# Service and added the necessary _CRT_SECURE_NO_WARNINGS.

The debug version of the service is registered using 'installutil.exe' tool.

I can get the debugger to break just before the line where the dll is entered via a call to System.Diagnostics.Debugger.Break();.

I found some instruction on the net regarding stepping into debugging unmanaged code, and enabled the 'Enable unmanaged code debugging' check box, I've also tried turning on the Options->Debugging->Native 'Load DLL exports' and 'Enable RPC Debugging' (even though it's not COM). I've also copied the debug dll and .pdb to the same bin directory as the service.

However the unmanaged code is not being stepped into which is what I really need.

UPDATE: I found the Debugging Type in the DLL properties and set it to 'Mixed' as per suggestion on several sites but to no avail.

UPDATE2: My project now emits the debug dll and the pdb to the same directory as the debug service. Still unable to debug the dll.

+2 A:

Try setting the unmanaged code as the startup project. I know it doesn't make sense but I remember this working for a very similar project.

Since the DLL doesn't have an associated executable, when you try to run it will pop up asking what app to run. Browse to your C# app and then you should be good to go.

Happy debugging!

EDIT: it's been a while, but I think the debugging type Mixed is correct

Sam Post 2010-03-26 11:52:07

It's a service - you can't run a service as an app.

graham.reeds 2010-03-26 11:54:35

Huh, I was wondering about that. I don't know much about services I'm afraid. How do you debug a service without the DLL issue?

Sam Post 2010-03-26 12:04:55

Also, if you can't get this working via any other means, you could write a very simple C# app just for debugging the DLL.

Sam Post 2010-03-26 12:13:09

Well I created a simple console app to do what the dll was failing to do correctly (create a unc path - instead it created a folder of the unc name in the root of c:). The simple test worked so I guess there is some other interaction going on in the dll. To use the actual dll is a no-go as it is a single file of 17952 lines of sparsely commented c code.

graham.reeds 2010-03-26 13:00:31

You set a breakpoint on your service right before you call into the DLL. So why not call the same function in DLL from console app? You can pass in the same exact arguments as the service and hopefully find the bug in your DLL

Sam Post 2010-03-26 14:39:30

I can't call the function because it needs a dozen other calls to set up all the resources it expects in place. By the time I've done that I've practically rewritten the service. I created a simple console app that did the unc directory creation and it worked so it must be some interaction within the dll of the inputs.

graham.reeds 2010-03-26 15:10:02

In the end I created a console app and recreated all the prior calls just to make sure the call would act as it did in the actual service with the actual parameters once it got there.

I chronicled my fix and the resultant code at my site.

graham.reeds 2010-04-08 20:43:02

ansaurus

tags:

views:

answers:

Debugging of native code

related questions