I need to detect when an URL redirects the browser to another location, including javascript redirects. I thought of using an actual web browser engine with javascript support to achieve this, by loading and interpreting the html from the url and setting some sort of "hook" to be notified whenever the browser changes the address. Can anyone tell me if that's possible and with what browser engine? Or if there's a better method doing this task?
+2
A:
You can't. There will always be a way around your code. Frames, framebusting, 202 status codes, iframes, ajax content loading, and more.
You can write code that covers most cases, but it won't be 100% effective. What's the reason you're trying to do this? Maybe there's a better solution.
guest
2009-11-20 16:02:41
I'm working on a webcrawler that's able to parse and follow links like those from url shorteners...
2009-11-20 17:04:42
A:
A possible [almost] post-facto way to "catch them all", is with a software http proxy. This proxy would "see" all url requests following a particular initial page, and could act accordingly.
Small problem may be to deal with some of the content that comes by reference (images and such), but a few heuristics can probably help categorize these two varieties of URLs.
mjv
2009-11-20 16:08:52