ansaurus

Question

Answer 1

+1 A:

First off, consider using the lxml implementation of ElementTree:
http://codespeak.net/lxml/
This is a wrapper for libxml2, which I find performs well.

Run python interactively, make multiple queries against the same etree object. ipython is an enhanced interactive python interpreter with easy access to introspection and convenience-syntax.

E.g. interactively examining note.xml with ipython using lxml.etree.

$ ipython
Python 2.5.1 (r251:54863, Jul 10 2008, 17:24:48)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from lxml import etree

In [2]: doc = etree.parse(open("note.xml"))

In [3]: etree.dump(doc.getroot())
<note>
        <to>Tove</to>
        <from>Jani</from>
        <heading>Reminder</heading>
        <body>Don't forget me this weekend!</body>
</note>
In [4]: doc.xpath('/note/*')
Out[4]:
[<Element to at 89cf02c>,
 <Element from at 89cf054>,
 <Element heading at 89cf07c>,
 <Element body at 89cf0a4>]

MattH 2010-03-24 13:25:25

A great suggestion, but not sure it will work for me as I actually want to run the python scripts from an applescript. I will see if I can make it work

Jack 2010-03-24 13:54:21

Answer 2

+2 A:

While I second the suggestion to use lxml, you can get a huge performance boost by using the builtin cElementTree.

from xml.etree import cElementTree as ElementTree

mikerobi 2010-03-24 13:48:24

Yes I think that will work! The first time a query is run it takes around 15 seconds, but subsequent queries are around 1-2 seconds.As for using lxml my reason for not doing so is that I want to make sure these scripts will work out of the box on any mac I run them on, without needing to install anything else. I don't know if there's a way I could build lxml and just bundle it together with my .py files, if so please let me know how I do this

Jack 2010-03-24 14:03:53

Answer 3

+1 A:

Seconding the lxml recommendation, look at this article for how to improve performance by using an iterative (SAX-like) parsing method. It can be a pain at first since it can turn really procedural and messy, but it makes things faster. As you can see from these benchmarks, lxml is most likely your best bet for performance.

Tom 2010-03-24 13:51:50

+1 Interesting article

MattH 2010-03-24 14:19:54

ansaurus

tags:

views:

answers:

Repeatedly querying xml using python

related questions