views:

34

answers:

4

I'm trying to debug a C/C++ native DLL project from Visual Studio 2010. I'm attempting to follow these instructions: http://msdn.microsoft.com/en-us/library/c91k1xcf(v=VS.100).aspx

I want to use the built-in debugger and be able to step code, examine structures, etc. as I would do with a regular .exe project. The instructions on the page above describe a Debugging category under Configuration Properties which I do not see.

htt p://img707.imageshack.us/img707/4402/lalasz.png (remove space)

Just pressing F5 to debug results in the following error:

Unable to start program 'C:\Users\.......Test.dll'

I've used the debugger for regular .exe projects many times and it works fine on this computer. I'm not sure if I'm just missing something very obvious right now though.

Edit: Since I didn't make it clear from the start, I want Visual Studio to LoadLibrary my DLL into a stub process and let me debug at a source level from there, much like how OllyDbg does it.

My DLL is not the type that holds a bunch of functions to be exported and called. Instead it does a switch/case in the DllMain and on DLL_PROCESS_ATTACH will spawn a new thread. Therefore all I need Visual Studio to do is to load my DLL into a stub executable and allow me to set breakpoints, etc.

A: 

You probably have as the startup project the one that produces the dll.

You have two choices: either change the startup project to another project that produces an executable that uses that DLL, or configure from project properties the debug settings for the dll project to start an external application that uses that dll (Project Properties/Debug/Command).

Cătălin Pitiș
+1  A: 

Native DLLs cannot be run standalone - they must run in the context of some program. See this part of the instructions page you referenced.

"If you start debugging from the project that creates the DLL, you must specify the executable you want to use in debugging the DLL."

Steve Townsend
+1  A: 

You right-clicked the solution name in the Solution Explorer window and got the solution properties. Note that the window says "Solution Test Property Pages".

Right-click the project name (Test in bold) instead to set the project options.

Hans Passant
Thanks Hans. The Debugging tab was there when I right-clicked the correct place but I've realised I misread the page anyway. I don't actually want to have to specify an executable. Ideally VC++ would LoadLibrary my DLL on a stub executable and let me step it this way. Much like how OllyDbg lets you do this. Do you know if this is possible ?
Mike
No, VS needs an .exe, any .exe, that loads the DLL and calls the entrypoint that you want to test. Beyond something like the ActiveX Test Container, there is no generic .exe that can figure out what arguments your exported function needs. A unit test is always a good choice.
Hans Passant
My DLL is not the type that contains functions to be exported and called. The entry point will switch/case on dwReason and on DLL_PROCESS_ATTACH will spawn a thread with CreateThread() which will do some work. The same DLL is able to debugged with a 'generic .exe' in OllyDbg which simply uses a stub exe which calls LoadLibrary on the DLL. I can easily create my own skeleton executable which does the same thing as well but I won't be able to debug it through the IDE.
Mike
Yes, use that exact same EXE. Yes, you can debug it, set a breakpoint on the thread function. The debugger will automatically activate the breakpoint as soon as the DLL is loaded.
Hans Passant
Hans, that method works fine. For reference, these are the settings I used ultimately :http://img839.imageshack.us/img839/770/lalan.pngI get an error 'No Debugging Information' when debugging where the error states :Debugging information for 'loaddll.exe' cannot be found or does not match. Binary was not built with debug information.I guess there is nothing I can do about this though and it does not harm me.
Mike
A: 

You would run into this problem from a Managed project also. What Visual Studio is telling you is that it cannot run the DLL, just the same as you cannot double-click a DLL from Explorer, and have a program run.

In order to debug the DLL, write a small console application which calls functions from your DLL and exercises your code. If your DLL has a function foo(), call foo() from main in your console application. Set the console application as the "Startup" project, by right clicking the project name in the Solution Explorer and selecting its option.

Then, when you press F5, you will run the console application, which will call the DLL.

maxwellb
My DLL will work slightly different to the conventional one. It will spawn a thread in DllMain and do other work from there. I'm aware this is bad practice. I want to be able to debug it as I would be able to from a disassembler like OllyDbg. With the difference that I can do so at a source level.
Mike