Sphinx + tokyo Tyrant + mysql

Question

I'm looking at creating a full text search engine for one of my projects. We have a Mysql, Tokyo Tyrant and file documents that need to be indexed.

I'm looking at Sphinx right now but I can't figured out if I can use it to index every document.

I know it's possible to let Sphinx to use Mysql but I'm looking at a way to let Sphinx index and query Tokyo Tyrant as well as index file documents.

It could be Sphinx or Xapian or another one but no JAVA (Lucene is out) but something that can be used with PHP and run on Linux.

Any idea of a search engine that can accept more that Mysql as the source?

Thanks

Answer 1

If you don't have too many documents, and want to use only PHP, you should take a look at Zend_Search_Lucene :

Zend_Search_Lucene is a general purpose text search engine written entirely in PHP 5. Since it stores its index on the filesystem and does not require a database server, it can add search capabilities to almost any PHP-driven website.

Still, if you are using your own server, you can probably install software, which means you can deploy Lucence/SolR of Sphinx, even if they are not written in only-PHP -- this might actually be a safer solution if you need to index a lot of content.

Answer 2

You can send data from any source to sphinx using the xmlpipe2

You'd just need to write some code (probably in PHP) to read from Tokyo Tyrant (not sure what that is so you're on your own there) and the filesystem and package it all up as XML. Then you just set up your sphinx source to run a command that executes your custom code and sphinx takes care of the rest.

Answer 3

Hello all,

Can anyone please tell what steps are needed to get Sphinx up and running? I have installed Sphinx and MySQL and I have a bunch of text files. They are all unformatted (no fields, no tables, just regular paragraphs of text). It seems that xmlpipe2 is the way to go, but for the life of me, I am not able to figure out how to get started.

I am a newbie to MySQL, so please bear with me if this seems too trivial.

Thanks a lot, in advance !!!

Sesh