There are two different things you may want:
1) prompting when a window is closed. You can use the "close" event, e.g.:
<!-- Creates an unclosable window -->
<window onclose="return false" ...>
I just checked it and it does work.
2) prompting when the application is about to quit. You can do this by observing the quit-application-requested notification via the nsIObserverService (example).
(1) is closer to what you asked, since when the user presses the [X] button on the window, the sequence of events is this:
- "close" event is dispatched to the window
- if nothing prevented it, the window closes
- if no windows are left, and it's not Mac OS X, the application starts to shut down, quit-application-requested is sent.
- If no observers deny the quit request, the shutdown proceeds.
On the other hand, if you put your check solely in the onclose handler of the window, other ways of quitting application (File-Exit on Win/-Quit on Mac) may trigger it too late (e.g. after closing other windows). Also, if your app can have more than one "main" window (with onclose logic), the user will have to deal with multiple confirmations, which may or may not be what you want.
Here are some references to the Firefox code (if I'm not missing something, it should also be packaged with your application), you can research and reuse the approach taken: