Logging into SO with curl

Question

I'm working on a project and I want to log into SO via curl.

I use Google as my openID provider which means that I need to log into Google first via its API.

Here is the code I have so far

#!/usr/bin/env sh
. ./params.sh #the script with $username and $password
curl --silent https://www.google.com/accounts/ClientLogin \
-d Email=$username -d Passwd=$password \
-d accountType=GOOGLE \
-d source=localhost-test-1 \
-d service=lso \
-o tokens
. ./tokens
echo $Auth; #$Auth is correct here - I did not get a BadAuth error.

endpoint="https://www.google.com/accounts/o8/id";

curl http://stackoverflow.com/users/authenticate \
    -d "openid_identifier=$endpoint" \
    -w %{redirect_url}> ./google_url
google_url=$(cat ./google_url);
echo $google_url;
echo;
echo;
echo;
curl -L --silent --header "Authorization: GoogleLogin auth=$Auth" $google_url;

At this point I get a page from google telling me that stack overflow wants information and I have to log in. According to http://code.google.com/apis/gdata/articles/using_cURL.html the --header ... $Auth part should count as a login and redirect me to stack overflow.

Here is the form I get when I run this script:

<form id="gaia_universallogin"
      action="https://www.google.com/accounts/ServiceLoginAuth?service=lso" method="post">
  <input type="hidden" name="continue" id="continue"
           value="https://www.google.com/accounts/o8/ud?st=SOME_KEY" />
  <input type="hidden" name="service" id="service"
           value="lso" />
  <input type="hidden" name="dsh" id="dsh"
           value="SOME_NEG_NUMBER" />
</form>

When I try the answer below I get the following error:

Can't call method "attr" on an undefined value at - line 8.
curl: (3) <url> malformed

-->

here is the output from google2.html

<form id="gaia_loginform"      
        action="https://www.google.com/accounts/ServiceLoginAuth?service=lso" method="post"                >
  <input type="hidden" name="continue" id="continue"            value="https://www.google.com/accounts/o8/ud?st=RNADOM" />
  <input type="hidden" name="service" id="service"            value="lso" />
  <input type="hidden" name="dsh" id="dsh"            value="NEG_NUMEBER" />
  <input type="hidden"              name="GALX"             value="ABCD" />
  <input type="text" name="Email"  id="Email" />
  <input type="password"   name="Passwd" id="Passwd" > 
  <input type="checkbox" name="PersistentCookie" id="PersistentCookie"    value="yes"
  <input type="hidden" name='rmShown' value="1" />
  <input type="submit" class="gaia le button" name="signIn" id="signIn"                 />
<input type="hidden" name="asts"    >
</form>

Answer 1

The Google login service is specific to the particular service you're using (Google docs vs Google Analytics vs Google Maps etc). The service ode you've specified (lh2) is specific to Google Picasa.

Unfortunately there doesn't seem to be an equivalent code for OpenId (at least, not that I could find!)

The page that you get back from Google should contain a login form. If you look at that, it should be possible to construct a curl invocation to log in; that should then redirect you back to SO (or whichever openID page you want to log in to) logged in.

It turns out that doing this is a little tricky, because you have to parse out some of the the form fields to submit them back to Google, and because Google doesn't send back a straight HTTP redirect, but an HTML doc with a <meta http-equiv="redirect" ...> tag. And of course you have to enable cookies. But this is all possible in a script using curl - the following works for me:

#!/bin/bash

# Utility function for parsing values out of an HTML form
get_value()
{
    local tagtype="$1" attrname="$2" attrvalue="$3" getattr="$4"
    perl -MHTML::TreeBuilder - "$@" <<EOF
         @args=@ARGV;
         \$h=HTML::TreeBuilder->new;
         \$h->parse_file("$htmlfile");
         while (\$#args > 0) {
             \$h=\$h->look_down(_tag => shift @args,
                                shift @args => shift @args);
         }
         print \$h->attr(shift @args);
EOF
}

# Empty the cookie jar
cj="cookiejar"
rm -f "$cj"

# Attempt to log in to SO. This will redirect to a google URL.
endpoint="https://www.google.com/accounts/o8/id"

google_url=`curl -L -s -S http://stackoverflow.com/users/authenticate \
    -d "openid_identifier=$endpoint" \
    -o /dev/null -b "$cj" -c "$cj" \
    -w %{url_effective}`
echo $google_url
echo
echo

# Retrieve the form from Google
htmlfile=googleform.html
curl -L -s -S -o "$htmlfile" -b "$cj" -c "$cj" "$google_url"

# Parse out the form fields
form_url=`get_value form id gaia_loginform action`

fdsh=`get_value form id gaia_loginform input name dsh value`
fcontinue=`get_value form id gaia_loginform input name continue value`
fservice=`get_value form id gaia_loginform input name service value`
fGALX=`get_value form id gaia_loginform input name GALX value`
frmShown=`get_value form id gaia_loginform input name rmShown value`
fsignIn=`get_value form id gaia_loginform input name signIn value`

fEmail='INSERT LOGIN EMAIL HERE'
fPasswd='INSERT PASSWORD HERE'

# Submit the login form
htmlfile=google2.html
curl -L -s -S -o "$htmlfile" -b "$cj" -c "$cj" --data-urlencode dsh="$fdsh" \
  --data-urlencode continue="$fcontinue" \
  --data-urlencode service="$fservice" \
  --data-urlencode GALX="$fGALX" \
  --data-urlencode Email="$fEmail" \
  --data-urlencode Passwd="$fPasswd" \
  --data-urlencode rmShown="$frmShown" \
  --data-urlencode signIn="$fsignIn" \
  "$form_url"

# Interpret the redirect
redirect=`get_value meta http-equiv refresh content | sed "s/^.*'\(.*\)'.*$/\1/"`

# Follow it
htmlfile=google3.html
curl -L -s -S -o "$htmlfile" -b "$cj" -c "$cj" "$redirect"

(Note that I seem to have a slightly different version of curl from you, so I had to change the -w options slightly; I'm guessing my version will work for you but you may need to tweak it.)