Python: Fast querying in a big dbf (xbase) file

Question

Hi all,

I have a big DBF file (~700MB). I'd like to select only a few lines from it using a python script. I've seen that dbfpy is a nice module that allows to open this type of database, but for now I haven't found any querying capability. Iterating through all the elements from python is simply too slow.

Can I do what I want from python in a reasonable time?

Answer 1

If you are using Windows you can use the odbc module in combination with the Visual FoxPro ODBC Driver

Answer 2

Chances are, your performance is more I/O bound than CPU bound. As such, the best way to speed it up is to optimize your search. You probably want to build some kind of index keyed by whatever your search predicate is.