For dialog applications traditional behavior has been to close the application when "Escape" is pressed. However some users complain about this as they often press the key by mistake.
What's your approach?
1) Close the application
2) Do nothing
3) Pop an "Are you sure you want to quit?" message (possibly with a "Don't show this again" check box)
That depends on the dialog - if the user's likely to have put any effort into the dialog, then (3) sounds like a good approach:
Pop an "Are you sure you want to quit?" message (possibly with a "Don't show this again" check box").
Otherwise, closing the dialog (not the application - surely?) is standard, and expected by many users.
Don't allow the user to lose significant amounts of work by accidentally pressing escape.
I would choose 1) when displaying information and there is no interaction between dialog and user.
And I would choose 3) if there is expected to be some kind of user interaction.
Option 2) is not a good option. USers expect something to happen when they click Escape button.
As Dominic says, using Esc to close an application, is considered by most a default function.
Remember, there are a lot more people using the Esc with a purpose, than people accidently clicking the Esc.
I think that closing the application on escape is a thing of the past when the UI was not that clear and you had only one method to abort.
Today with the close button and the alt-F4 (or apple-C on the mac if I'm not mistaken) there is not need to such UX any more.