Subclassing a window from a thread in c#

Question

I'm creating a thread that looks for a window. When it finds the window, it overrides its windowproc, and handles WM_COMMAND and WM_CLOSE.

Here's the code that looks for the window and subclasses it:

public void DetectFileDialogProc()
{
   Window fileDialog = null;
   // try to find the dialog twice, with a delay of 500 ms each time
   for (int attempts = 0; fileDialog == null && attempts < 2; attempts++)
   {
      // FindDialogs enumerates all windows of class #32770 via an EnumWindowProc
      foreach (Window wnd in FindDialogs(500))
      {            
         IntPtr parent = NativeMethods.User32.GetParent(wnd.Handle);
         if (parent != IntPtr.Zero)
         {
            // we're looking for a dialog whose parent is a dialog as well
            Window parentWindow = new Window(parent);
            if (parentWindow.ClassName == NativeMethods.SystemWindowClasses.Dialog)
            {
               fileDialog = wnd;
               break;
            }
         }
      }
   }
   // if we found the dialog
   if (fileDialog != null)
   {
      OldWinProc = NativeMethods.User32.GetWindowLong(fileDialog.Handle, NativeMethods.GWL_WNDPROC);
      NativeMethods.User32.SetWindowLong(fileDialog.Handle, NativeMethods.GWL_WNDPROC, Marshal.GetFunctionPointerForDelegate(new WindowProc(WndProc)).ToInt32());
   }
}

And the windowproc:

public IntPtr WndProc(IntPtr hWnd, uint msg, IntPtr wParam, IntPtr lParam)
{
   lock (this)
   {
      if (!handled)
      {
         if (msg == NativeMethods.WM_COMMAND || msg == NativeMethods.WM_CLOSE)
         {       
            // adding to a list. i never access the window via the hwnd from this list, i just treat it as a number
            _addDescriptor(hWnd);
            handled = true;
         }
      }
   }
   return NativeMethods.User32.CallWindowProc(OldWinProc, hWnd, msg, wParam, lParam);
}        

This all works well under normal conditions. But I am seeing two instances of bad behavior in order of badness:

1. If I do not close the dialog within a minute or so, the app crashes. Is this because the thread is getting garbage collected? This would kind of make sense, as far as GC can tell the thread is done? If this is the case, (and I don't know that it is), how can I make the thread stay around as long as the dialog is around?

2. If I immediately close the dialog with the 'X' button (WM_CLOSE) the app crashes. I believe its crashing in the windowproc, but I can't get a breakpoint in there. I'm getting an AccessViolationException, The exception says "Attempted to read or write protected memory. This is often an indication that other memory is corrupt." Its a race condition, but of what I don't know. FYI, I had been reseting the old windowproc once I processed the commands, but that was crashing even more often!

Any ideas on how I can solve these issues?

Answer 1

Two points of observation that I can make....

• In your DetectFileDialogProc, you are comparing wnd to null, that is an IntPtr type yes? if so, that check for the comparison should be if (wnd > IntPtr.Zero){ .... }
• In your WndProc, you are using the this variable for the lock which is a bad thing to do...you should do something like this private readonly object objLock = new object(); and within your WndProc use this lock (objLock){....}

and see if that resolves the issue....

Answer 2

Finally came up with a solution, attacking the problem from a different angle. I was able to set a system-wide hook in managed code using SetWinEventHook, and the option WINEVENT_OUTOFCONTEXT, which amazingly has the property: The callback function is not mapped into the address space of the process that generates the event. I trap the event EVENT_SYSTEM_DIALOGSTART to receive notifications whenever a dialog is created, and EVENT_SYSTEM_DIALOGEND when its destroyed.