mechanize

Custom auth handler with mechanize

I would like to use python-ntlm with mechanize.Browser() I have got HTTPNtlmAuthHandler working with urllib2 and mechanize.urlopen() and tried to use it with Browser() but it doesn't work Here is the code I'm using for urlopen passman = mechanize.HTTPPasswordMgrWithDefaultRealm() passman.add_password(None, url, user, password) auth_NTL...

mechanize and redirecting (ruby)

I thought that mechanize follows redirection by default ... by my script ends at the redirection page. How can I handle this? require 'rubygems' require 'mechanize' agent = WWW::Mechanize.new page = agent.get("http://www.vbulletin.org/forum/index.php") login_form = page.form_with(:action => 'login.php?do=login') login_form['vb_logi...

Use getControl to control objects other than the name variable

Hello, I am using the Zope testbrowser which has been recommended in my last question. The problem that I am facing is that I can use the getControl function to control different objects like: password, username etc. I am trying to submit the page to get to the next page but the submit button has no 'name' variable, just an 'id' variab...

installing mechanize with easy_install

I just got easy_install downloaded but i'm having problems installing mechanize, should I be addressing site-packages at any point. In the first try below, i got an error. in the second try below, i got command not found which is wierd since I know for sure that it downloaded. names-computer:~ names$ cd /Users/names/Desktop/ names-compu...

bug or desired behaviour? couldn't identify form using string 'post' but 'POST'. html contains 'post' though (ruby,mechanize)

the code that didn't work: login_form = page.form_with(:method => 'post') and code that works login_form = page.form_with(:method => 'POST') I inspected the form object via puts page.forms.inspect and got [#<WWW::Mechanize::Form {name nil} {method "POST"} ....] html source: <form class="login" method="post"> <fieldset> <legend>...

log in with browser and then ruby/mechanize takes it over?

is that even possible? what I need to pass to mechanize? With what url I can start then? I cannot manage (so far) to log into one website using mechanize so I was thinking if I can do this little workaround. I believe I can capture all cookies and everything else and then pass it to ruby/mechanize to do the rest ... screenshots below a...

how to capture and then pass cookies to ruby/mechanize ( a workaround to log in)

this questions is linked with my other question log in with browser and then ruby/mechanize takes it over? I am not able (hopefully I will in near future :-) to log into one website using ruby/mechanize so I was thinking if there is any workaround for the log in part. After that I want my script to take over and do its job. Could some...

How to add cookie to existing cookielib CookieJar instance in Python?

I have a CookieJar that's being used with mechanize that I want to add a cookie to. How can I go about doing this? make_cookie() and set_cookie() weren't clear enough for me. br = mechanize.Browser() cj = cookielib.LWPCookieJar() br.set_cookiejar(cj) ...

`post form` parameters difference between firefox and mechanize (ruby/mechanize)

I am trying to figure out if mechanize sends correct post query. I want to log in to a forum (please see html source, mechanize log in my other question) but I get only the login page again. When looking into it I can see that firefox sends out post with parameters like auth_username=myusername&auth_password=mypassword&auth_login=Log...

how to add new field to mechanize form (ruby/mechanize)

there is a public class method to add field to mechanize form I tried .. #login_form.field.new('auth_login','Login') #login_form.field.new('auth_login','Login') and both gives me an error undefined method "new" for #<WWW::Mechanize::Form::Field:0x3683cbc> (NoMethodError) I tried login_form.field.new('auth_login','Login') which give...

Ruby - Mechanize: Select link by classname and other questions

Hello! At the moment I'm having a look on Mechanize. I am pretty new to Ruby, so please be patient. I wrote a little test script: require 'rubygems' require 'mechanize' agent = WWW::Mechanize.new page = agent.get('http://www.google.de') pp page.title google_form = page.form_with(:name => 'f') google_form.q = 'test' page = agent.subm...

ruby mechanize in Facebook

I'm trying to click the Settings button on the home page, but when I do I get this page back: #<WWW::Mechanize::Page {url #<URI::HTTP:0x1023c5fc0 URL:http://www.facebook.com/editaccount.php?ref=mb&amp;drop&gt;} {meta} {title nil} {iframes} {frames} {links} {forms}> which is.. kinda empty! Is there some problems with these iframes ...

python mechanize proxy question

I've got mechanize setup and working with python. I am adding support for using a proxy, but how do I check that I am actually using the proxy? Here is some code I am using: ip = 'some proxy ip address' br.set_proxies({"http://": ip} ) I started to wonder if it was working because just to do some testing I typed in: ip = 'asdfasd...

Mechanize not recognizing anchor tags via CSS selector methods

(Hope this isn't a breach of etiquette: I posted this on RailsForum, but I haven't been getting much response from there recently.) Has anyone else had problems with Mechanize not recognizing anchor tags via CSS selectors? The HTML looks like this (snippet with white space removed for clarity): <td class='calendarCell' align='left'> <...

Why can't WWW::Mechanize::Firefox locate it's "new" method?

I get the following error when I run the little sample script: Can't locate object method "new" via package "WWW::Mechanize::Firefox" (perhaps you forgot to load "WWW::Mechanize::Firefox"?) at mechtest.pl line 2. use WWW::Mechanize::Firefox; my $mech = WWW::Mechanize::Firefox->new(); $mech->get('http://google.com'); $mech->eval_in_pag...

Is the python mechanize cache feature the same as normal browser cache feature?

In its document, it says "This caching behaviour can be avoided by using Mechanize.OpenerDirector". Does this cache influence next request of the same page? ...

UnicodeDecodeError problem with mechanize

Hi, I receive the following string from one website via mechanize: 'We\x92ve' I know that \x92 stands for ’ character. I'm trying to convert that string to Unicode: >> unicode('We\x92ve','utf-8') UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 2: unexpected code byte What am I doing wrong? Edit: The reason I t...

How do I get the text-form verification code when doing auto site access in Perl?

I'm playing around with Win32::IE:Mechanize to try to access some authentication-required sites automatically. So far I've achieved moderate success, for example, I can automatically log in to my yahoo mailbox. But I find many sites are using some kind of image verification mechanism, which is possibly called CAPTCHA. I can do nothing to...

How to get 'value' of select tag based on content of select tag, using Nokogiri

How would one get the contents of the 'value' attribute of a select tag, based on content of the select tag (i.e. the text wrapped by option), using Nokogiri? For example, given the following HTML: <select id="options" name="options"> <option value="1">First Option - 4</option> <option value="2">Second Option - 5</option> <op...

Problem with Ruby mechanize and inheritance

Hi, I'm working with mechanize and having problems with inheritance when testing in a rails environment, using script/console. When I write like this: require 'rubygems' require 'mechanize' agent = WWW::Mechanize.new agent.get 'http://www.google.com' Everything works fine. But when I try to create a subclass of WWW::Mechanize like ...