I think there are two parts to this question: how to hook up your own exception handler so as to allow the app to continue, and whether it's possible to reuse the Windows Forms unhandled exception UI.
For the first part, see Application.DispatcherUnhandledException. If you subscribe to this event, and your event handler sets DispatcherUnhandledExceptionEventArgs.Handled to true, then WPF will skip default unhandled exception processing -- i.e. the app will not automatically shut down. (Of course your event handler can still shut it down.) Handled is not set to true by default so you must do this explicitly.
For the second part, see System.Windows.Forms.ThreadExceptionDialog. This is officially "not intended for use from your code," and is not documented in any useful way, so it's not wise to rely on it for production applications. However if you are willing to take a chance then you can create an instance of this class and ShowDialog() it. In .NET 3.5, it returns DialogResult.Cancel to mean "ignore exception and continue" and DialogResult.Abort to mean "quit." These values are not documented and should be treated as implementation details though!