How do I fix the "Reentrancy was detected" warning given by the Visual Studio debugger?

Question

When running our unit tests in debug mode, at a certain point the Visual Studio debugger breaks to show the reentrancy MDA. The linked article explains that this occurs when A low-level operating system extensibility point, such as the vectored exception handler calls back into managed application code.

Apparently this can cause heap corruption or other serious problems, so I would definitely like to fix it.

I am looking at the stacktrace at the point where this warning is shown, but I'm having trouble figuring out which "low-level operating system extensibility point" is involved here. I don't see any unmanaged/managed transitions other than the once caused by mstest and by calling the System.Windows.Forms.Cursors.VSplit getter. Also, simply calling that getter from a unit test doesn't seem to be sufficient to trigger the warning.

What have I done wrong here, and how do I fix it?

Answer 1

As Hans Passant suggested (though unfortunately he has now deleted his answer here), the problem is that in my unit test I'm invoking code which is only designed to be called in the context of a GUI thread with a message loop. There is no such message loop in a unit test.

In this case the problem is the System.Windows.Forms.Cursors.VSplit getter. I see two ways to avoid this:

• strictly follow the MVVM pattern and keep the problematic code separate in the view, only test the viewmodel.
• or hide the cursor managment code behind an interface which is mocked in the unit test