How to click a link that has javascript:__doPostBack in href?

Question

I am writing a screen scraper script in python with module 'mechanize' and I would like to use the mechanize.click_link() method on a link that has javascript:__doPostBack in href. I believe the page I am trying to parse is using AJAX.

Note: mech is the mechanize.Browser()

>>> next_link.__class__.__name__
'Link'
>>> next_link
Link(base_url='http://www.citius.mj.pt/Portal/consultas/ConsultasDistribuicao.aspx', url="javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')", text='2', tag='a', attrs=[('id', 'ctl00_ContentPlaceHolder1_Pager1_lnkNext'), ('title', 'P\xc3\xa1gina seguinte: 2'), ('href', "javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')")])
>>> req = mech.click_link(next_link)
>>> req
<urllib2.Request instance at 0x025BEE40>
>>> req.has_data()
False

I would like to retrieve the page source after clicking the link.

Answer 1

I don't think mechanize supports Javascript; to scrape pages which intrinsically rely on Javascript execution for their functionality, you may need to use a different tool, such as Selenium RC.

Answer 2

next_link.class.name 'Link' next_link Link(base_url='http://www.citius.mj.pt/Portal/consultas/ConsultasDistribuicao.aspx', url="javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')", text='2', tag='a', attrs=[('id', 'ctl00_ContentPlaceHolder1_Pager1_lnkNext'), ('title', 'P\xc3\xa1gina seguinte: 2'), ('href', "javascript:__doPostBack('ctl00$ContentPlaceHolder1$Pager1$lnkNext','')")]) req = mech.click_link(next_link) req req.has_data() False