What are situation when you want to use window.showModalDialog function? It seams that you can do exactly the same with window.open function and few parameters that remove some of the chrome (navigation, addressbar, etc...)
When would you want to use window.showModalDialog and window.open?
Modal dialogs are dialogs that once opened by the parent, do not allow you to focus on the parent until the dialog is closed.
One could use a modal dialog for a login form, edit form, etc where you want to have a popup for user interaction but not allow the user to return to the window that opened the popup.
As a side note, I believe only Internet Explorer implementes window.showModalDialog, so that kind of limits your usage of it.
Note that there's a bug in Chrome 2 which prevents showModalDialog() from loading properly. The popup window appears, but the content never loads.
One more reason to avoid using showModalDialog().