Python and text manipulation

Question

Hi,

I want to learn a text manipulation language and I have zeroed in on Python. Apart from text manipulation Python is also used for numerical applications, machine learning, AI, etc. My question is how do I approach the learning of Python language so that I am quickly able to write sophisticated text manipulation utilities. Apart from regular expressions in the context of "text manipulation" what language features are more important than others what modules are useful and so on.

Thanks.

Answer 1

I found the object.__doc__ and dir(obj) commands incredibly useful in learning the language.

e.g.

a = "test,test,test"

What can I do with a? dir(a). Seems I can split a.

vec = a.split (",")

What is vec? vec.__doc__:

"new list initialized from sequence's items"

What can I do with vec? dir(vec).

vec.sort ()

etc ...

Answer 2

read standard python tutorial http://docs.python.org/tutorial/
after that "Dive into Python" http://diveintopython.org/

also for training you could use Project Euler ( http://projecteuler.net ) and PythonChalange ( http://www.pythonchallenge.com/ )

but speed of learning will depend from your programming skills in general=)

for me functional programming features was more important, they make my code most elegant.

Answer 3

Beyond regular expressions here are some important features:

• Generators, see Generator Tricks for Systems Programmers by David Beazley for a lot of great examples to pipeline unlimited amounts of text through generators.

For tools, I recommend looking at the following:

• Whoosh, a pure Python search engine that will give you some nice real life examples of parsing text using pyparsing and text processing in Python in general.

• Ned Batcheldor's nice reviews of various Python parsing tools.

• mxTextTools

• Docutils source code for more advanced text processing in Python, including a sophisticated state machine.

Edit: A good links specific to text processing in Python:

• Text Processing in Python by David Mertz. I think the book is still available, although it's probably a bit dated now.

Answer 4

There's a book Text Processing in Python. I didn't read it myself yet but I've read other articles of this author and generally they're a good staff.