Any recommended VC++ settings for better PDB analysis on release builds

Question

Are there any VC++ settings I should know about to generate better PDB files that contain more information?

I have a crash dump analysis system in place based on the project crashrpt.

Also, my production build server has the source code installed on the D:\, but my development machine has the source code on the C:\. I entered the source path in the VC++ settings, but when looking through the call stack of a crash, it doesn't automatically jump to my source code. I believe if I had my dev machine's source code on the D:\ it would work.

Answer 1

You could trying using the MS-DOS subst command to assign your source code directory to the D: drive.

Answer 2

In case anyone is interested, a co-worker replied to this question to me via email:

Artem wrote:

There is a flag to MiniDumpWriteDump() that can do better crash dumps that will allow seeing full program state, with all global variables, etc. As for call stacks, I doubt they can be better because of optimizations... unless you turn (maybe some) optimizations off.

Also, I think disabling inline functions and whole program optimization will help quite a lot.

In fact, there are many dump types, maybe you could choose one small enough but still having more info http://msdn.microsoft.com/en-us/library/ms680519(VS.85).aspx

Those types won't help with call stack though, they only affect the amount of variables you'll be able to see.

I noticed some of those dump types aren't supported in dbghelp.dll version 5.1 that we use. We could update it to the newest, 6.9 version though, I've just checked the EULA for MS Debugging Tools -- the newest dbghelp.dll is still ok to redistribute.

Answer 3

Is Visual Studio prompting you for the path to the source file? If it isn't then it doesn't think it has symbols for the callstack. Setting the source path should work without having to map the exact original location.

You can tell if symbols are loaded by looking at the 'modules' window in Visual Studio.

Assuming you are building a PDB then I don't think there are any options that control the amount of information in the PDB directly. You can change the type of optimizations performed by the compiler to improve debuggabilty, but this will cost performance -- as your co-worker points out, disabling inline will help make things more obvious in the crash file, but will cost at runtime.

Depending on the nature of your application I would recommend working with full dump files if you can, they are bigger, but give you all the information about the process ... and how often does it crash anyway :)

Answer 4

Is Visual Studio prompting you for the path to the source file?

No.

If it isn't then it doesn't think it has symbols for the callstack. Setting the source path should work without having to map the exact original location.

Symbols are loaded successfully. It shows the callstack, but double clicking on an entry doesn't bring me to the source code. I can of course search in files for the line in question, but this is hard work :)

Answer 5

"Are there any VC++ settings I should know about"

Make sure you turn off Frame pointer ommision. Larry osterman's blog has the historical details about fpo and the issues it causes with debugging.

Symbols are loaded successfully. It shows the callstack, but double clicking on an entry doesn't bring me to the source code.

What version of VS are you using? (Or are you using Windbg?) ... in VS it should defintely prompt for source the first time if it doesn't find the location. However it also keeps a list of source that was 'not found' so it doesn't ask you for it every time. Sometimes the don't look list is a pain ... to get the prompt back up you need to go to solution explorer/solution node/properties/debug properties and edit the file list in the lower pane.

Finally you might be using 'stripped symbols'. These are pdb files generated to provide debug info for walking the callstack past FPO, but with source locations stripped out (along with other data). The public symbols for windows OS components are stripped pdbs. For your own code these simply cause pain and are not worth it unless you are providing your pdbs to externals. How would you have one of these horrible stripped pdbs? You might have them if you use "binplace" with the -a command.

Good luck! A proper mini dump story is a godsend for production debugging.

Answer 6

If your build directly from your sourcecode management system, you should annotate your pdb files with the file origins. This allows you to automatically fetch the exact source files while debugging. (This is the same proces as used for retrieving the .Net framework sourcecode).

See http://msdn.microsoft.com/en-us/magazine/cc163563.aspx for more information. If you use subversion as your SCM you can check out the SourceServerSharp project.

Answer 7

This is the procedure I used after some trouble similar to yours:

a) Copied to the production server all the EXE & DLL files that were built, each with its corresponding PDB to the same directory, started the system, and waited for the crash to happen.

b) Copied back all the EXE, DLL & PDB files to the development machine (to a temporary folder) along with the minidump (in the same folder). Used Visual Studio to load the minidump from that folder.

Since VS found the source files where they were originally compiled, it was always able to identify them and load them correctly. As with you, in the production machine the drive used was not C:, but in the development machine it was.

Two more tips:

• One thing I did often was to copy an EXE/DLL rebuilt and forget to copy the new PDB. This ruined the debug cycle, VS would not be able to show me the call stack.

• Sometimes, I got a call stack that didn't make sense in VS. After some headache, I discovered that windbg would always show me the correct stack, but VS often wouldn't. Don't know why.