Straight Java/Groovy versus ETL tool (Talend/etc) - what libraries would you use?

Question

Assume you have a small project which on the surface looks like a good match for an ETL tool like Talend.

But assume further, that you have never used Talend and furthermore, you do not trust "visual programming" tools in general and would rather code everything the old fashioned way (text on a nice IDE!) with the help of an appropriate language & support libraries.

What are some language patterns & support libraries that could help you stay away from the ETL tool temptation/trap?

Answer 1

Depending on the size of the DB schema, you could map everything real quick in Hibernate and just use the resulting object model to do your work (depending on what you want the ETL tool for anyways)

Answer 2

I think this is a pretty good match for Rails-inspired frameworks, such as Grails on Groovy or Lift on Scala.

Answer 3

It depends on whether the deliverable is the processor or the output itself. If you just need to deliver the output, you don't need to maintain the code. If the code needs to be maintained then will it be you maintaining it or somebody else?

If somebody else needs to maintain I'd use Java or give them Talend.

If it's throwaway code, I'd use what will be easier or fun to program with.

If you need to maintain it and the processing is complex, I'd use Scala. It has:

• some libraries to interact with databases
• xml literals
• parser combinators
• interesting features on its collection packages (map, filter, groupBy, partition, ...)
• and of course any other existing Java libraries.