I have an application that is using about 100k more of the Desktop Heap in this version then it did last version. Is there a way I can see what is on the Desktop Heap and how big the individual objects are? Using Dheapmon I was able to see what percentage of the heap I was using, but I want more details.
You can walk the heap using the Win32 API call HeapWalk. You can call GetProcessHeap to get all the heaps available to the process if you need to walk more than just the default heap.
Dheapmon is the only tool I know of for looking directly at the desktop heap, but have you tried looking at your application with a tool like Winspector to look for glaring differences between the two versions (say, some type of control in your application now contains far more windows)? Any chance the application has switched to a newer version of IE? I seem to remember IE7 being much more desktop heap-intensive than IE6...
Stolen from a comment on a blog post here
Let me give a little background on how desktop heap allocations are made. The desktop heaps are in kernel mode virtual address space, so individual desktop heap allocations have to be made by a component running in kernel mode. In particular, win32k.sys is the only kernel mode component that makes desktop heap allocations. win32k.sys in the kernel mode side of Win32, and it includes both the window manager (USER) and GDI. It is the window manager piece of win32k.sys that uses desktop heap. The functionality of the window manager is exposed to processes running in user mode through user32.dll. It is user32.dll that exports user mode callable functions that are implemented in win32k.sys. So if a process does not load user32.dll, it will not use desktop heap.
Regarding your question about setting a breakpoint that will catch desktop heap allocations... yes, there is such a function - win32k!DesktopAlloc. However, this is a kernel mode function, and to set a breakpoint on it will require that you use a kernel debugger.
That sounds all scary complicated to me who has never ventured away from user-mode in Windows.
When I had a similar problem I just put break points all over the startup portion of our application. At each break I would watch the level of allocated handles and what dhelpmon told me. Doing a sort of binary search I started to narrow down where the allocations were happening.
Dheapmon definitely loads a kernel-mode driver and does some other complicated stuff to get its information about desktop heaps. I don't think you can replicate what it does with GetProcessHeap and HeapWalk.
In Vista/Server2008/Windows 7 then the Task Manager shows some useful information about handle counts, USER objects, etc that could help track down what kind of resources are being used. In older versions of Windows some of the same information is shown in the Process Explorer tool from sysinternals.