mechanize

Mechanize in Jquery

I am adept in Ruby WWW::Mechanize. And I was wondering if a jQuery equilavent exists ? ...

trying to POST with ruby mechanize

I've captured the login HTTP headers using firefox plugin LiveHTTPheaders. I've found the following url and variables. POST /login email=myemail%40gmail.com&password=something&remember=1&loginSubmit=Login And here's the code I am running: require 'rubygems' require 'mechanize' browser = Mechanize.new browser.post('http://www.mysit...

WWW::Mechanize and Cookies

I use WWW::Mechanize::Shell to test stuff. Since I didn't managed to sign in on a web site I want to scrape, I thought I will use the browser cookie (chrome or firefox) for that specific website with the 'cookie' command WWW::Mechanize::Shell has. The question is, Cookies usually stored in a single file, which is not good, how to get a ...

How do I manually add more cookies to a session which already has cookies set in mechanize?

I have a python script which scrapes a page and receives a cookie. I want to append another cookie to the existing cookies that are being send to the server. So that on the next request I have the cookies from the original page plus ones I set manually. Anyway of doing this? I tried addheaders in mechanize but it was ignored. ...

Can python's mechanize use localhost sites?

Can Mechanize access sites being locally hosted by Apache? ...

how do I manually add cookies to WWW::Mechanize's already existing cookiejar

I have a perl script where after fetching a page I need to add a cookie to an already existing cookiejar with cookies in it already. how do I go about doing this? I'm hoping for a python mechanize style .set_cookie() function ...

how do I set a field with a duplicate name of another field in mechanize?

I am trying to submit a form with 2 fields with the same name but different type. I can identify the correct field I want by the field type or the number. Whats the best way of setting the correct field without iterating through all the fields? ...

in mechanize, is there anyway of rewriting the url to POST to in a form?

I'm running a python script and I'm using mechanize. The form i'm trying to submit normally uses javascript to rewrite the url to POST to, so to correctly submit the form i need to manually do the same. is there anyway of doing this? ...

How to deploy this "Python+twill+mechanize" combination to "Google App Engine"?

Hello, I've been trying to pass my login and password from Python script to the eBay sign-in page. Later I want this script to be run from "Google App Engine" I was suggested to use "mechanize". Unfortunately, it didn't work for me: IDLE 1.2.4 >>> import re >>> import mechanize >>> br = mechanize.Browser() >>> br.open("https:...

Does GAE accept twill at all?

Hello, I have created my GAE application in directory "my_application". Inside this directory I created a .py file and named it "my_scrypt". The contents of "my_scrypt" in the beginning were as following: print 'Content-Type: text/plain' print '' print 'This is my first application' Then I ran it locally on my machine (Windows X...

python's mechanize wont properly parse a form

I'm trying to submit a form using python's mechanize but it wont properly parse the form in question. There are 4 other forms, which are parsed correctly except for this one form. The form is properly parsed in perl's www::mechanize though but i'd like to stick with python. Is there anyway of retrieving the html of the page and editing...

retrieve application/json document with twill/mechanize in authenticated session

Hi, I need to retrieve a document with MIME type "application/json". I'm using twill to log in to a site and when I attempt to go to the URL pointing to the JSON document and show it, I get this message: 'The HTTP header field "Accept" with value "text/html; */*" could not be parsed.' I have tried changing the "Accept" field to "app...

How to select a Radio Button?

I am using mechanize and I am trying to select a button from a radio button list. This list has 5 items. How can I select the first item? Docs didn't help me. >>> br.form <ClientForm.HTMLForm instance at 0x9ac0d4c> >>> print(br.form) <form1 POST http://www.example.com application/x-www-form-urlencoded <HiddenControl(DD=17010200) (readon...

python mechanize javascript submit button problem!

Hello All im making some script with mechanize.browser module. one of problem is all other thing is ok, but when submit() form,it not working, so i was found some suspicion source part. in the html source i was found such like following. im thinking, loginCheck(this) making problem when submit form. but how to handle this kind of...

Scraping Flash: accessing background files, perhaps in Mechanize?

Hi there I'm scraping a website in Flash, writing in Python. I can see in Firebug that the page loads its Flash file and then some background data in an .asmx file. The background data is what I'm interested in - so how can I get hold of the .asmx file? I already know what it's called. I can't get at the .asmx file directly, but can ...

Encoding problem downloading HTML using mechanize and Python 2.6

browser = mechanize.Browser() page = browser.open(url) html = page.get_data() print html It shows some strange characters. I suppose that it is UTF-8 string but Python doesn't know that and cannot show it properly. How can I convert this string to unicode string like u = u'test' ...

Python mechanize: read table element next to input

I'm using mechanize to fill out a form that has a series of rows. Each row is an <input type="checkbox"> followed by <td>name of the checkbox</td>. How do I read the name so I know whether to check the box? Thanks ...

can mechanize read ajax? (ruby)

can I get the correct data/text that is displayed via AJAX using mechanize in ruby? Or is there any other scripting gem that would allow me to do so? ...

Python Mechanize, setting proxy for mech browser? This way isn't working

I'm trying to use a proxy with Python's mechanize automated browser library but it doesn't seem to be working. I am trying the following and checking the output and its not getting the page. I have made sure I am testing with proxies that work as I test the proxy manually in my browser first. Any advice on proper syntax is appreciated as...

How can I login and download a file with Perl's WWW::Mechanize?

I'm trying to use Perl's WWW::Mechanize to download a file. I have to login the website before and then, after having validated the form, download the file. The thing is, after hours, I didn't succeed doing what I want. At the end, the script save a file which is not a zip file but a html file with nothing interesting in it. Here is th...