Mechanize and JavaScript

Question

Hello folks; i'm new with ruby and coding just for fun. i just found WWW::Mechanize and i loved it at the first time, There is my question: i'm connecting to a web site, logging in. website redirects me to new pages and mechanize deals with all cookie and redirection jobs. bu i cant get the last page. i used firebug and did same job again and saw that there are two more pages i had to pass with Mechanize. i take a quick look to the pages and saw that there is some javascript&html code but couldn't understand (it doesn't look like normal page codes) :( what are those pages for? how they can redirect to other pages. what should i do to pass these?

excuse my english please :( thanks for every answer.

Answer 1

If you need to handle pages with JS, try WATIR - it drives a real web browser, and can thus handle any Javascript. Mind you, it requires either IE or Firefox with a certain extension installed, and you will see the pages flash on the screen as it works.

Your other option would be understanding what the Javascript on the offending page does and bypassing it manually.

Answer 2

At presesnt, Mechanize doesn't handle Javascript. There's talk of eventually merging Johnson's capabilities into Mechanize, but until that happens, you have two options:

1. Figure out the javascript well enough to understand how to traverse those pages.
2. Automate an actual browser that does understand Javascript using Watir.

Answer 3

what are those pages for? how they can redirect to other pages. what should i do to pass these?

Sometimes work is done on those pages. Sometimes the javascript is there to prevent automated access like what you're trying to do :). A lot of websites have unnecessary checks to make sure you have a "good" browser, so make sure that your user_agent is set to something common, like IE. Sometimes setting the user_agent to look like an old browser will let you get past without javascript.

Website automation is fun because you have to outsmart the website and its software developers, using multiple strategies. Like the others said, Watir is the best tool for getting past javascript atm.

Answer 4

thank you everybody. watir have done just what i wanted to do :) it looks great, takes me more into ruby :)

at first time the website i was trying to get was angry with me because of user_agent, but when i set it to firefox, problem has gone.

stackoverflow rocks! i love here :)