PostMessage and CALLWNDHOOK, missing registered messages?

Question

Here's the situation. I've got a two different window hooks, one's a global hook of the WH_SHELL variety that watches for new top-level windows, the second's a thread hook of the WH_CALLWNDPROC which is set on some of the top-level windows discovered by the first hook. The two hooks are implemented in different DLLs.

As far as I can tell, both hooks are being installed properly. However, nothing I Post to the hooks windows with a message code > WM_USER a register message is ever handled by the custom WH_CALLWNDPROC hook, but "normal" windows messages are passing thought it just fine.

Code that hooks a discovered window:

... Getting handle, mod, and procHook ...
DWORD threadId = GetWindowThreadProcessId(handle, NULL);

HHOOK hook = SetWindowsHookEx(WH_CALLWNDPROC, (HOOKPROC)procHook, mod, threadId);

if(!PostMessage(handle, CUSTOM_MESSAGE, NULL, NULL))
{
  ... fetch and print error message ...
}

The body of the hook itself:

... Report sends a message to an agreed upon window with the passed wParam & lParam
Report(20, nCode);

if(nCode == CUSTOM_MESSAGE)
{
  ... This code is never reached ...
  Report(50, ERROR_SUCCESS);

  if(PerformTask())
    Report(200, ERROR_SUCCESS);
  else
    Report(400, ERROR_SUCCESS);
}

... More code handling more messages in the same basic form

The first Report call is what's confirmed that the hook is installed and working, as it posts back a bunch of messages in the low teens and twenties (ERASEBACKGROUND, PAINT, etc.).

CUSTOM_MESSAGE is defined as WM_USER + 314. The message used for Report(...) is WM_USER + 317.

*I have since updated my code to use RegisterWindowMessage to obtain a UINT to send, it was incorrect of me to use WM_USER for inter-process communication.*

So, basically, what is wrong with either my design or my usage of windows hooks and PostMessage? If I've omitted any details, let me know; there's alot of code involved and this is a pretty large question already, so I've tried to only include what I think is relevant.

As an aside, is there an agreed upon better way to debug hooks? I've been using the moral equivalent to cout << ... everything by posting messages to an agreed upon window and debugging its WndProc.

Thanks,
-Kevin Montrose

Answer 1

While @Michael is correct regarding the use of WM_USER messages (they should be used within an application only - registered messages are the better way to go here), at the same time I think the reason that you aren't receiving them is due to the nature of the CallWndProc hook and posted messages. I'm not positive, but I think that you want to hook the GetMessage hook for posted messages.

Another option is to hook the Debug hook, which receives all messages before all other hooks. You could have that watch for your custom message, and then from there determine which hook (if any) received your custom message.

EDIT: For debugging hooks, just attach to the process in question, and set a breakpoint in your hook procedure. That ends up being the easiest way I've found to debug them.

Answer 2

I think I figured it out. Simple case of not quite absorbing the documentation.

My CallWndProc hook behaved as if nCode, wParam, and lParam were what would be passed to the hooked threads WndProc. In fact, lParam contained a pointer to a CWPSTRUCT; read the data out of that structure and everything works fine.