open source data mining/text analysis tools in python

Question

I have a database full of reviews of various products. My task is to perform various calculation and "create" another "database/xml-export" with aggregated data. I am thinking of writing command line programs in python to do that. But I know someone have done this before and I know that there is some open source python solution or similar which probably gives lot more interesting "aggregated data" then I can possibly think off.

The problem is I don't really know much about this area other then basic data manipulation from command line nor I know what are the terms I should use to even search for this thing.. I am really not looking for some scientific/visualization stuff (not that I don't mind if the tool provides), something simple to start with and gradually see/develop stuff what I need.

My only requirement is either the "end aggregated data" be in a database or export as XML file no proprietary stuff. Its a bit robust then my python scripts as I have to deal with "lots" of data across 4 machines.

Any hint where should I start my research?

Thanks.

Answer 1

What kind of analysis are you trying to do?

If you're analyzing text take a look at the Natural Language Toolkit (NLTK).

If you want to index and search the data, take a look at the whoosh search engine.

Please provide some more detail on what kind of analysis you're looking to do.

Answer 2

Looks like you are looking for a Data Integration solution.
One suggestion is the open source Kettle project part of the Pentaho suite.
For python, a quick search yielded PyDI and SnapLogic