In fact, you can actually do what you really want to do, which is bust the iframe buster. The technique lets you use onbeforeunload to switch the page back to yours, but indirectly, since the browsers are too smart to let you set the URL in onbeforeunload. So instead, onbeforeunload sets an indicator variable to mark that the URL has changed, and you periodically poll that variable using a setInterval routine established when your page loads. As long as you're polling fast enough to catch the variable change, you can jump in and change the page's URL yourself. The trick is to change it to a page that returns a 204, a special status which tells the browser to leave the current page alone.
That said, this is a cat-and-mouse game. Check out Jeff Attwood's StackOverflow question on this, where he asks how to bust the above technique. Web pages can bust the buster buster by beating the poll interval; basically, they set the URL to point to a tiny page, and one that has already been cached. As soon as the URL changes to that page, it will load faster than the poll routine can jump in and notice that the indicator variable has changed.
It's not easy to prevent loading the frame in the first place. If you really wanted, you could have your server download the page and parse the Javascript to see if the iframe-busting technique is present. However, short of emulating a browser, you can only rely on basic pattern-matching and it would be easy for a page to bypass that. (e.g. use top["l"+"ocation"] instead of top.location).
A smarter technique would be to track which URLs were redirecting using Ajax requests back to the server. (e.g. if the iframe is still there after it has loaded, send an Ajax request back to your server). You can't 100% guarantee the accuracy of those requests, since they come from the browser, but you can at least use them to build up a manual blacklist.
You also can't force the iframe to break out into another window.