HttpClient login, search and get the xml content

Question

I want to login to a site using HttpClient and after logging in I want to search for something and retrieve the contents of the search result.

/** * A example that demonstrates how HttpClient APIs can be used to perform * form-based logon. */ public class TestHttpClient {

public static void main(String[] args) throws Exception {

    DefaultHttpClient httpclient = new DefaultHttpClient();

    HttpGet httpget = new HttpGet("http://projecteuler.net/");

    HttpResponse response = httpclient.execute(httpget);
    HttpEntity entity = response.getEntity();

    System.out.println("Login form get: " + response.getStatusLine());
    if (entity != null) {
        entity.consumeContent();
    }
    System.out.println("Initial set of cookies:");
    List<Cookie> cookies = httpclient.getCookieStore().getCookies();
    if (cookies.isEmpty()) {
        System.out.println("None");
    } else {
        for (int i = 0; i < cookies.size(); i++) {
            System.out.println("- " + cookies.get(i).toString());
        }
    }

    HttpPost httpost = new HttpPost("http://projecteuler.net/index.php?section=login");

    List <NameValuePair> nvps = new ArrayList <NameValuePair>();
    nvps.add(new BasicNameValuePair("IDToken1", "username"));
    nvps.add(new BasicNameValuePair("IDToken2", "password"));

    httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));

    response = httpclient.execute(httpost);

    System.out.println("Response "+response.toString());
    entity = response.getEntity();

    System.out.println("Login form get: " + response.getStatusLine());
    if (entity != null) {

        InputStream is = entity.getContent();
        BufferedReader br = new BufferedReader(new InputStreamReader(is));
        String str ="";
        while ((str = br.readLine()) != null){
            System.out.println(""+str);
        }
    }

    System.out.println("Post logon cookies:");
    cookies = httpclient.getCookieStore().getCookies();
    if (cookies.isEmpty()) {
        System.out.println("None");
    } else {
        for (int i = 0; i < cookies.size(); i++) {
            System.out.println("- " + cookies.get(i).toString());
        }
    }
    httpclient.getConnectionManager().shutdown();        
}

}

when I print the output from HttpEntity ... its printing the login page contents.

How do I get the contents of the page after I login using HttpClient ?

Answer 1

The post should mimick the form submit. No need to get the login page first. If I take a look at http://projecteuler.net, it seems the form is posted to index.php, so I'd try changing the post url:

HttpPost httpost = new HttpPost("http://projecteuler.net/index.php");

Use something like Fire bug to see what is exactly happening in the browser. Maybe you should follow a redirect after logging in (HttpClient supports this). There also seems to be a parameter called "login"with value "Login" that is being posted.

Answer 2

Hi I am having the same issue:

below is the html code of the login page :

////////////////////////////////////////////////////////////////////

FORM method="post" name="loginForm" action="/cvg/dispatch/login/submit;jsessionid=48734C3B961C73F68C9404BD594FBF3B"

///////////////////////////////////////////////////////////////////

Below is my java program:

public static void main(String[] args) throws Exception {

    DefaultHttpClient httpclient = new DefaultHttpClient();

    HttpGet httpget = new HttpGet("http://matrix.itasoftware.com/cvg/dispatch");

    HttpResponse response = httpclient.execute(httpget);
    HttpEntity entity = response.getEntity();

    System.out.println("Login form get: " + response.getStatusLine());
    if (entity != null) {
        entity.consumeContent();
    }
    System.out.println("Initial set of cookies:");
    List<Cookie> cookies = httpclient.getCookieStore().getCookies();
    if (cookies.isEmpty()) {
        System.out.println("None");
    } else {
        for (int i = 0; i < cookies.size(); i++) {
            System.out.println("- " + cookies.get(i).toString());
        }
    }

    HttpPost httpost = new HttpPost("http://matrix.itasoftware.com/cvg/dispatch/login/submit;");

    List <NameValuePair> nvps = new ArrayList <NameValuePair>();
    nvps.add(new BasicNameValuePair("loginValues.userName", "myuserName"));
   // nvps.add(new BasicNameValuePair("IDToken1", "myuserName"));
  //  nvps.add(new BasicNameValuePair("IDToken2", "mypassword"));
    nvps.add(new BasicNameValuePair("loginValues.password", "mypassword"));

    httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));

    response = httpclient.execute(httpost);
    entity = response.getEntity();
    System.out.println("Login form get: " + response.getStatusLine());
    if (entity != null) {
        BufferedReader br = null;
        br = new BufferedReader(new InputStreamReader(entity.getContent()));
        String readLine;
        while(((readLine = br.readLine()) != null)) {
          System.err.println(readLine);
        }


        //response.getEntity().getContent();
      //  entity.consumeContent();
    }