Gui reentrancy with managed waiting

Question

Hello,

I've found a reentrancy problem when using NotifyIcons. It's really easy to reproduce, just drop a NotiftIcon on a form and the click event should look like this:

private bool reentrancyDetected;
private void notifyIcon1_MouseClick(object sender, MouseEventArgs e)
{
    if (reentrancyDetected) MessageBox.Show("Reentrancy");
    reentrancyDetected = true;
    lock (thisLock)
    {
        //do nothing
    }
    reentrancyDetected = false;
}

Also start a background thread which will cause some contention:

private readonly object thisLock = new object();
private readonly Thread bgThread;
public Form1()
{
    InitializeComponent();
    bgThread = new Thread(BackgroundOp) { IsBackground = true };
    bgThread.Start();
}

private void BackgroundOp()
{
    while (true)
    {
        lock (thisLock)
        {
            Thread.Sleep(2000);
        }
    }
}

Now if you start clicking on the notifyicon the message will pop up, indicating the reentrancy. I'm aware of the reasons why managed waiting in STA should pump messages for some windows. But i'm not sure why the messages of notifyicon are pumped. Also is there a way to avoid the pumping without using some boolean indicators when entering/exiting mehtods ?

Any help is appreciated thanks.

Answer 1

You can see what's happening if you replace the MessageBox.Show call by Debugger.Break and attach a debugger with native debugging enabled when the break hits. The call stack looks like this:

WindowsFormsApplication3.exe!WindowsFormsApplication3.Form1.notifyIcon1_MouseClick(object sender = {System.Windows.Forms.NotifyIcon}, System.Windows.Forms.MouseEventArgs e = {X = 0x00000000 Y = 0x00000000 Button = Left}) Line 30 + 0x1e bytes   C#
System.Windows.Forms.dll!System.Windows.Forms.NotifyIcon.OnMouseClick(System.Windows.Forms.MouseEventArgs mea) + 0x6d bytes 
System.Windows.Forms.dll!System.Windows.Forms.NotifyIcon.WmMouseUp(ref System.Windows.Forms.Message m, System.Windows.Forms.MouseButtons button) + 0x7e bytes   
System.Windows.Forms.dll!System.Windows.Forms.NotifyIcon.WndProc(ref System.Windows.Forms.Message msg) + 0xb3 bytes 
System.Windows.Forms.dll!System.Windows.Forms.NotifyIcon.NotifyIconNativeWindow.WndProc(ref System.Windows.Forms.Message m) + 0xc bytes 
System.Windows.Forms.dll!System.Windows.Forms.NativeWindow.Callback(System.IntPtr hWnd, int msg = 0x00000800, System.IntPtr wparam, System.IntPtr lparam) + 0x5a bytes  
user32.dll!_InternalCallWinProc@20()  + 0x23 bytes  
user32.dll!_UserCallWinProcCheckWow@32()  + 0xb3 bytes  
user32.dll!_DispatchClientMessage@20()  + 0x4b bytes    
user32.dll!___fnDWORD@4()  + 0x24 bytes 
ntdll.dll!_KiUserCallbackDispatcher@12()  + 0x2e bytes  
user32.dll!_NtUserPeekMessage@20()  + 0xc bytes 
user32.dll!__PeekMessage@24()  + 0x2d bytes 
user32.dll!_PeekMessageW@20()  + 0xf4 bytes 
ole32.dll!CCliModalLoop::MyPeekMessage()  + 0x30 bytes  
ole32.dll!CCliModalLoop::PeekRPCAndDDEMessage()  + 0x30 bytes   
ole32.dll!CCliModalLoop::FindMessage()  + 0x30 bytes    
ole32.dll!CCliModalLoop::HandleWakeForMsg()  + 0x41 bytes   
ole32.dll!CCliModalLoop::BlockFn()  - 0x5df7 bytes  
ole32.dll!_CoWaitForMultipleHandles@20()  - 0x51b9 bytes    
WindowsFormsApplication3.exe!WindowsFormsApplication3.Form1.notifyIcon1_MouseClick(object sender = {System.Windows.Forms.NotifyIcon}, System.Windows.Forms.MouseEventArgs e = {X = 0x00000000 Y = 0x00000000 Button = Left}) Line 32 + 0x14 bytes   C#

The relevant function is CoWaitForMultipleHandles. It ensures that the STA thread cannot block on a sync object without still pumping messages. Which is very unhealthy since it is so likely to cause deadlock. Especially in the case of NotifyIcon since blocking the notification message would hang the tray window, making all icons inoperative.

What you see next is the COM modal loop, infamous for causing re-entrancy problems. Note how it calls PeekMessage(), that's how the MouseClick event handler gets activated again.

What's pretty stunning about this call stack is that there's no evidence of the lock statement transitioning into code that calls CoWaitForMultipleHandles. It is somehow done by Windows itself, I'm fairly sure that the CLR doesn't have any provisions for it. At least not in the SSCLI20 version. It suggests that Windows actually has some built-in knowledge of how the CLR implements the Monitor class. Pretty awesome stuff, no clue how they could make that work. I suspect it patches DLL entrypoint addresses to revector the code.

Anyhoo, these special counter-measures are only in effect while the NotifyIcon notification runs. A workaround is to delay the action of the event handler until the callback is completed. Like this:

    private void notifyIcon1_MouseClick(object sender, MouseEventArgs e) {
        this.BeginInvoke(new MethodInvoker(delayedClick));
    }
    private void delayedClick() {
        if (reentrancyDetected) System.Diagnostics.Debugger.Break();
        reentrancyDetected = true;
        lock (thisLock) {
            //do nothing
        }
        reentrancyDetected = false;
    }

Problem solved.