It already works this way by default. The UI thread exception handling method is controlled by Application.SetUnhandledExceptionMode(). The default is UnhandledExceptionMode.CatchException so that the ThreadException event is raised and, by default, creates a ThreadExceptionDialog.
However, if a debugger is attached then it overrides this mode. So that an exception will always be unhandled if there is no active catch clause. So that the debugger will stop, allowing you to diagnose the problem. By writing your own try/catch, you prevent this from working.
Do beware that OnPaint() can be special, particularly for PictureBox. It has a try/catch clause, catching an unhandled exception and painting a red cross. This is a bit unusual but necessary because it supports the ImageLocation property. Which lets it display images from a potentially unreliable network source. The best way to trouble-shoot exceptions in that case is with Debug + Exceptions, tick the Thrown checkbox. This forces the debugger to always stop on an exception, even if it isn't unhandled.