haystack's RealTimeSearchIndex causes django to hang on data entry

Question

I'm using django-haystack and a xapian backend with real time indexing (haystack.indexes.RealTimeSearchIndexing) of model data and it works fine on my Ubuntu server. However, it causes django to hang upon data entry when I deployed the app on a RHEL5 server.

Everything is hunky dory if I switch to a standard SearchIndex.

Running ./manage.py rebuild_index manually works fine too.

The major differences between the two setups would be the versions of Python (2.4.3 vs 2.6.4) and the xapian (1.0.4-1 vs 1.0.15).

Any suggestions on what may be the problem?

Nothing interesting appears in the logs, and I've tried different databases (mysql, sqlite3) and deployment methods (mod_python, wsgi) with no luck yet.

I have noted the warning on the haystack docs stating that RealTimeSearchIndex is only handled gracefully with a Solr backend, however I'm running a very traffic site with only occasional writes so I'm fine with some CPU overheads on writes.

Answer 1

Please note this from the author of xapian-haystack:

Because Xapian does not support simultaneous WritableDatabase connections, it is strongly recommended that users take care when using RealTimeSearchIndex to either set WSGIDaemonProcess processes=1 or use some other way of ensuring that there are not multiple attempts to write to the indexes. Alternatively, use SearchIndex and a cronjob to reindex content at set time intervals (sample cronjob can be found here http://gist.github.com/216247_) or derive your own SearchIndex to implement some other form of keeping your indexes up to date.

Answer 2

Installing xapian-core and xapian-bindings from source solved the problem.

I initially used the RPM packages provided here.