Easy way to process batch data through web service?

Question

What simple tools would you recommend to read a text file of addresses, send each record separately to a web service for geocoding, and save the batch of results as a text file?

Looking for no-frills component(s) with usage examples, for minimal code-from-scratch. Language irrelevant as long as dev environment is easy to install.

Requirements - usable by unsophisticated programmer - low or no cost - runs under Windows.

---

Second thought: How easy would this be to do inside a browser using JavaScript and a library or two?

Answer 1

Given the generic nature of the requirements, the relatively simple workflow, but its potential to bring a few twists and turns in the design (for example, the need of using https rather than http for webservices, the need of producing some odd token for authentication, or some fancy marshaling or conversion etc.) it might be best to use a modern script language. A very basic plan could be to use a plain shell script (a bat file), base on curl and a few other command line utilities, but this approach may not be flexible enough to deal with some requirements; instead languages such as Perl, PHP, Python, Ruby would be much preferable.

This would provide a low entry barrier, the ability to test elements of the application interactively before putting them into a formal script, and to leverage extensive libraries to deal with the various requirements that may arise, such as the storage of configuration parameters, parsing detail, output format, webservices, maths associated with geo positions etc. etc.

My inclination would be to use Python, but as said most other modern dynamic languages would do.

Answer 2

I would use XMLunit with Eclipse IDE, + JUnit, and JDK1.6 . A finished program that does this might only be 100 lines of code. It's doable by someone who is a novice programmer...

When the program is done you can compile as an .exe file for future use.

I would choose "Strawberry Perl" as my second choice for programming language. Python is slightly harder to use I think.

Answer 3

I'd go for Java and use a flat file parsing library like jFFP or Flatworm These libraries are pretty easy to understand and to use (I've worked with both of them in the past) and they both provide code samples.

Spring Integration would be another good option but the learning curve might be too big if you are not familiar with Spring and it might be overkill for your simple workflow.

Actually, in your case, I think I'd choose Flatworm for the parsing. You'll find code samples on its website or in How to read and parse flat files in Java. And you could even use it to write your output file like in Writing flat files in Java with Flatworm).

For the SOAP part, I'd use the JAX-WS Reference Implementation (which is included in the JDK 6 so you won't have to add any library if you are using Java 6) and Netbeans IDE. Netbeans IDE has very good support for developing JAX-WS Web Services Client (or here for later versions of Netbeans) and should really ease the process. Once the various classes generated, calling the web service is a matter of 3 lines of code as shown in the examples of the provided links:

// Call Web Service Operation
com.cdyne.ws.Check service = new com.cdyne.ws.Check();
com.cdyne.ws.CheckSoap port = service.getCheckSoap();

// TODO initialize WS operation arguments here
java.lang.String bodyText = "";
java.lang.String licenseKey = "";

// TODO process result here
com.cdyne.ws.DocumentSummary result = port.checkTextBody(bodyText, licenseKey);