HttpClient POST fails to submit the form + resulting string is cut-off (incomplete)

Question

Hi,

I'm writing an app to check for the bus timetable's. Therefor I need to post some data to a html page, submit it, and parse the resulting page with htmlparser.

Though it may be asked a lot, can some one help me identify if 1) this page does support post/get (I think it does) 2) which fields I need to use? 3) How to make the actual request?

this is my code so far:

String url = "http://busspur02.aseag.de/bs.exe?Cmd=RV&Karten=true&DatumT=30&DatumM=4&DatumJ=2010&ZeitH=&ZeitM=&Suchen=%28S%29uchen&GT0=&HT0=&GT1=&HT1=";
            String charset = "CP1252";
            System.out.println("startFrom: "+start_from);
            System.out.println("goTo: "+destination);

            //String tag.v

            List<NameValuePair> params = new ArrayList<NameValuePair>();
            params.add(new BasicNameValuePair("HTO", start_from));
            params.add(new BasicNameValuePair("HT1", destination));
            params.add(new BasicNameValuePair("GTO", "Aachen"));
            params.add(new BasicNameValuePair("GT1", "Aachen"));
            params.add(new BasicNameValuePair("DatumT", day));
            params.add(new BasicNameValuePair("DatumM", month));
            params.add(new BasicNameValuePair("DatumJ", year));
            params.add(new BasicNameValuePair("ZeitH", hour));
            params.add(new BasicNameValuePair("ZeitM", min));

            UrlEncodedFormEntity query = new UrlEncodedFormEntity(params, charset);

            HttpPost post = new HttpPost(url);
            post.setEntity(query);
            InputStream response = new DefaultHttpClient().execute(post).getEntity().getContent();

            // Now do your thing with the facebook response.
            String source = readText(response,"CP1252");
            Log.d(TAG_AVV,response.toString());
            System.out.println("STREAM "+source);

EDIT:
This is my new code:

try {
            HttpClient client = new DefaultHttpClient();  
            String getURL = "http://busspur02.aseag.de/bs.exe?SID=5FC39&ScreenX=1440&ScreenY=900&CMD=CR&Karten=true&DatumT="+day+"&DatumM="+month+"&DatumJ="+year+"&ZeitH="+hour+"&ZeitM="+min+"&Intervall=60&Suchen=(S)uchen&GT0=Aachen&T0=H&HT0="+start_from+"&GT1=Aachen&T0=H&HT1="+destination+"";
            HttpGet get = new HttpGet(getURL);
            HttpResponse responseGet = client.execute(get);  
            HttpEntity resEntityGet = responseGet.getEntity();  
            if (resEntityGet != null) {  
                //do something with the response
                Log.i("GET RESPONSE",EntityUtils.toString(resEntityGet));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }   

But the output file is cut-off. If I do the same request in a browser I get like 14 different routes. Now the file suddenly stops and I only get 3 routes.... what's wrong?

I solved the last problem with the cut-off string: click here to see my solution

Answer 1

String url = "http://busspur02.aseag.de/bs.exe?Cmd=RV&Karten=true&DatumT=30&DatumM=4&DatumJ=2010&ZeitH=&ZeitM=&Suchen=%28S%29uchen&GT0=&HT0=&GT1=&HT1=";

That form submits by GET. You should also submit by GET. You also need to gather as many input fields (<input>, <select>, <textarea>, <button>, etc, also those of type="hidden"!) from the HTML source and specify them as parameters of your request as well. A common thing which is been overlooked in that kind of automated form submits is the name/value pair of the submit button.

This one:

<input TYPE="Submit" accesskey="s" class="SuchenBtn" name="Suchen" tabindex="20" VALUE="(S)uchen">

You need to add at least Suchen=(S)uchen to your query string. That's the only way for the server side to find out if any submit button was pressed and if so, which one, so that it can take action accordingly.

Answer 2

Guten Tag!

In the following extract:

<td class="Start3"> 
          <input type="text" name="GT0" value="" tabindex="1" />
</td

The "class=start3" is instructing the browser to format a table antry in a particular way. What you are interested in is the "&GT0=your text snippet on the "GET" url generated from this form.