You can create a base class that will be an interface for your modal class. Your modal class will inherit from this base class. The class(es) using the modal class will only know that base class.
For the unit tests, you implement another class, dedicated solely to unit testing, based on the base class (interface) and offering a controllable behavior. For instance, your unit test can create a class that will always return as if OK (or Cancel or Help) has been clicked. Or this test class can be made parameterizable.
This class will return immediately when asked to display the modal window so that the unit test won't stop.
The code receives a reference (or a pointer) to the base class, which will be an instance of the modal class in production, and an instance of the mock during unit testing.
The technique of passing a test class offering the same interface as the real class instead of an instance of the real class is known under the Dependency Injection name.
Look for M. Feathers' "the humble dialog box" article.