I'm using Mechanize to try and login to http://fedex.com but doing so keeps returning this error: 405 => Net::HTTPMethodNotAllowed Is that a response from fedex.com? Any ideas how to get around it? Here's my Mechanize code: agent = Mechanize.new { |a| a.log = Logger.new("mech.log") } agent.user_agent_alias = 'Mac Safari' page = agent...
I have a webpage which redirects to another url in the form of http://www.example.com/url.html#midpage. I'm wondering if there's anyway for WWW::Mechanize to follow http://www.example.com/url.html instead of http://www.example.com/url.html#midpage? ...
Here LINK it is suggested that it is possible to "Figure out what the JavaScript is doing and emulate it in your Python code: " This is what I would like help doing ie my question. How do I emulate javascript:__doPostBack ? Code from a website (full page source here LINK: <a style="color: Black;" href="javascript:__doPostBack('ctl00$Co...
Hi, I'm trying to figure out a way to transfer cookies from a Twill browser instance to a Zope browser instance. They both are built on Mechanize, so this should be possible. I have tried: zopeBrowser = zope.testbrowser.browser.Browser() twillBrowser = twill.commands.get_browser() twillBrowser.go("http://example.com/") # got some coo...
So I have an authenticated site that I want to access via the mechanize module. I'm able to log in, and then go to the page I want. However, because the page recognizes that mechanize doesn't have javascript enabled, it wants me to click a submit button to get redirected to a non javascript part of the site. How can I simply click the bu...
i started this script in Calibre. when i found out that Calibre can not do what i want, i installed spyder and i am now in the process of making it real python. i am trying (at this stage) to get a list of url from an index. in the new urls, i want to get other urls (sort of an index of indexs) in order to get to the 2ns level index i n...
I'm using WWW::Mechanize::Firefox to crawl pages that load some JavaScript after they have been loaded. My code regarding this problem: my ($firemech) = WWW::Mechanize::Firefox->new(tab => 'current', ); $firemech->get($url); die "Cannot connect to $url\n" if !$firemech->success(); print "I'm connected!\n"; my ($retries) = 10; while ($...
I'm not sure what's going on, as the script used to work (before I messed around with my python on my system...) But when I try something along the lines of import mechanize browser = mechanize.Browser() browser.open("http://google.com") I get something like <response_seek_wrapper at 0x10123fd88 whose wrapped object = <closeable_res...
I'm having issues submitting the result of a form submission (I can submit a form, but I can't submit the form on the page that follows the first). I have: browser = mechanize.Browser() browser.set_handle_robots(False) browser.open('https://www.example.com/login') browser.select_form(nr=0) browser.form['j_username'] = 'username' brows...
I have been Googling for sometime but I guess I am using the wrong set of keywords. Does anyone know this URI that lets me request permission from Facebook to let me crawl their network? Last time I was using Python to do this, someone suggested that I look at it but I couldn't find that post either. ...
I'm coding a web crawler and I've been using WWW::Mechanize::Firefox to navigate some pages (for the others I use WWW::Mechanize) which keep loading content after the page loaded and I've never had an issue with that. Yesterday I added DBI and DBD::mysql to the script, adding queries to export data to a database (this works perfectly), ...
Is there any way to invoke fragment identifiers from python? I'm currently using python mechanize. ...
Hello, i am getting this error: >>> br = Browser() >>> br.open("http://www.bestforumz.com/forum/") <response_seek_wrapper at 0x21f9fd0 whose wrapped object = <closeable_response at 0x21f9558 whose fp = <socket._fileobject object at 0x021F5F30>>> >>> br.select_form(nr=0) Traceback (most recent call last): File "<pyshell#3>", line 1, i...