views:

133

answers:

1

Is this Python script correct?


import urllib, urllib2, cookielib 

username = 'myuser' 
password = 'mypassword' 

cj = cookielib.CookieJar() 
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) 
login_data = urllib.urlencode({'username' : username, 'j_password' : password}) 
opener.open('http://www.example.com/login.php', login_data) 
resp = opener.open('http://www.example.com/hiddenpage.php') 
resp.read()

I found this script HERE.It is meant to login to a webpage first, retrieve the cookies, store them and use them in order to open some other page in the same website. I want to login in this way to my eBay account (the URL is https://signin.ebay.com/ws/eBayISAPI.dll?SignIn ) and then go to my inbox on my eBay account (the URL is http://my.ebay.com/ws/eBayISAPI.dll?MyEbay&gbh=1) .

So, here are the values that I need to use in this script:

First (Sing-in) URL: https://signin.ebay.com/ws/eBayISAPI.dll?SignIn

Second URL: http://my.ebay.com/ws/eBayISAPI.dll?MyEbay&gbh=1

My login name on eBay: tryinghard

My password on eBay: gettingsomewhere

With all these new values the above script must look this way:


import urllib, urllib2, cookielib 

username = 'tryinghard' 
password = 'gettingsomewhere' 

cj = cookielib.CookieJar() 
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) 
login_data = urllib.urlencode({'username' : username, 'j_password' : password}) 
opener.open(https://signin.ebay.com/ws/eBayISAPI.dll?SignIn', login_data) 
resp = opener.open(http://my.ebay.com/ws/eBayISAPI.dll?MyEbay&gbh=1') 
resp.read()

Is it correct? I am especially suspicious about the login_data = line (the fourth one from bottom), why is it a j_password there instead of just password?

I tried this script with all these values and it didn't work. Does anybody know why it doesn't work in my case?

I've already learned how to log in to my eBay account and then check some other pages there by means of running a python script that is using twill as an external module, but that was only successful when I ran that script from the command prompt or from the Python shell. It wasn't successful when I tried running that script by means of "Google App Engine Software Development Kit" that I had downloaded from "Google App Engine".

Later I was told here that it wasn't successful because "Google App Engine" doesn't like external modules. That's why I found this script - those modules that it is importing in the very beginning (urllib, urllib2, cookielib) are all built-in modules.

+3  A: 

A simple "view source" on the login page whose URL you give reveals very easily the following detail about it... (just formatting the HTML minimally for readability):

<span style="display:-moz-inline-stack" class="unl">
  <label for="userid">User ID  </label></span>
<span><input size="27" maxlength="64" class="txtBxF"
       value="" name="userid" id="userid"></span></div>
<div><span style="display:-moz-inline-stack" class="unl">
  <label for="pass">Password  </label></span>
<span><input size="27" maxlength="64" class="txtBxF"
       value="" name="pass" id="pass" type="password"></span>

As you can see at a glance, the names of the crucial input fields are not username and j_password as you're using, but rather userid and pass. It's therefore obviously impossible for your code to work as it currently stands.

Read a bit more of the page and you'll also see soon after:

<input type="checkbox" name="keepMeSignInOption" value="1" id="signed_in"></b>
<span class="pcsm"><label for="signed_in"><b>Keep me signed in for today.</b>

Most likely you'll have to simulate that checkbox being selected to get cookies that are usable (at least for anything but a fleeting time;-).

And so on, and so forth, really -- the attempt to automate interaction with a page without bothering to read that page's source to get the actual IDs and names to use strikes me as definitely displaying a very optimistic attitude towards life, the universe, and everything...;-). Incidentally, to simplify such interaction (after perusing the source;-), I've found mechanize quite handy (and more robust than trying to hack it just with the standard library, as you are doing).

Also, before automatic interaction with a site, always check out its robots.txt to make sure you're not breaking its terms of use -- sites can easily identify "robots" (automated interaction) as opposed to "humans", and retaliate against robots.txt violation by banning, blacklisting, and worse; you don't really want to run into that;-).

Alex Martelli
Hello, Alex!!! It's so good to see You again!!! Yes, I admit I was overly naive, but somewhere in the back of my head I did expect someone to remind me of this need to read the page's source. Well, it seems now that the only way that is left for me is use mechanize. I afraid, though, that with mechanize I won't go any further than with twill (can run on my machine, but not on "GAE") - both are external modules and both can be equally denied by "GAE". In case the same happens with mechanize, do You think there is any way for me to "peek into" it to see how it is (please see my next comment)
brilliant
doing it and later repeat it in the (form of a) script?BTW, I just read the robots.txt link and I don't think that what I am going to do will violate eBay rules. These words confirm that: "Notwithstanding the foregoing, eBay may permit automated access to access certain eBay pages but solely for the limited purpose of including content in publicly available search engines" My purpose is just to check my own account there on a regular basis, nothing more than that. I think all kinds of automatic bidders that are being used on eBay, but were not issued by eBay, are way more dangerous.
brilliant
@brilliant, you _can_ run `mechanize` on GAE with simple fixes, cfr http://stackoverflow.com/questions/1902079/python-mechanize-gaepython-code/2056543#2056543 . I personally agree w/you wrt "automatic bidders", but that's just my personal opinion, after all, and I'm in no way affiliated with ebay nor was I ever, so my opinion about it is worth just about how much you pay for it;-).
Alex Martelli
I see, Alex. Thank You very much for this link. So, I am about to plunge myself into the world of mechanize!!!
brilliant
"...and I'm in no way affiliated with ebay nor was I ever, so my opinion about it is worth just about how much you pay for it"- :):):)
brilliant
Hello, Alex!!! I am sorry to bother You again. Can You imagine, I just succeeded in logging-in to eBay and Yahoo sign-in pages by using Python and twill!!!!! I was able to do that because one supporter on "Stackoverflow" instructed me to perform some changes in my "C:\Python25" folder. To me this way looks rather unusual and, perhaps, a little bit strange (nevertheless it works!) Plus, it is different, I guess, from what You referred to as "simple fixes". Now I have a dilemma: I don't know if this way would allow me to deploy my script to GAE. So, if You have time and desire (↙)
brilliant
would You please check out this page: http://stackoverflow.com/questions/3670701/how-to-deploy-this-pythontwillmechanize-combination-to-google-app-engine I described everything there in detail. Thank You.
brilliant
@brilliant, looks like the code is all pure python, so the only issue w/deploying it on App Engine is having it, **NOT** in site-packages (from which of course GAE's dev_appserver.py deploys absolutely nothing, nada, zilch) but in your GAE project's directory (I suggest a recursive zip of all the .py files, only -- forget the .pyc files, in particular). Let's continue on that other Question's thread, though, it's not all that relevant to this one.
Alex Martelli
Alex, to my huge regret, I have to admit here that my idea (of having a script on GAE that would log in to my eBay account by itself) has proved to be unfeasible. I've been able to deploy the script (with some of those simple fixes) and the script was really trying to log in, but instead of getting into my eBay account it would be taken back to eBay home page. And I didn't get any error messages on GAE, which means that everything was all right with the code - it is simply eBay doing its job on reverting such codes.
brilliant