how to import mysql tables to SOLR

Question

i can never understand how solr works.

it just talks about schema files all the way but how do i import content from the database to it with a painless method?

i have tried to figure it out by reading their tutorials but it just mess up my head.

its written for the Einsteins out there cause apparently there are a lot of people who also have difficulty of understanding it.

and they keep talking about the example folder. "just type java -jar ./start.jar". i mean..is this an example or can you use it as final for your website? where is data-config.xml located??? there are just no good tutorials out there explaining to that first time beginners can understand.

Answer 1

For folks who don't know what Solr is, it's part of the Apache Lucene project. It's a server that runs within a container such as Tomcat. Solr hosts a Lucene index and provides a "REST-like" interface to update and query an index via HTTP.

The "Getting Started" tutorial talks about starting the Solr server with java -jar start.jar, but that's only to get the Solr server running. It's like starting an instance of MySQL Server -- necessary before you can query it, but this step alone doesn't populate it with data or make it serve up any results.

The tutorial goes on to show an example of posting documents to the Solr server:

user:~/solr/example/exampledocs$ java -jar post.jar solr.xml monitor.xml

That example posts two documents, solr.xml and monitor.xml to be indexed. You don't have to use their post.jar example -- since Solr supports HTTP, you should be able to use any HTTP client, such as curl.

To index the entire result of an SQL query this way, you would have to write a script to loop over the result and post data to Solr row-by-row, but this would probably be excessively time-consuming, making a separate HTTP POST request per row of data.

I'm guessing the faster way is to use Solr's support for batch data in CSV format. See http://wiki.apache.org/solr/UpdateCSV for examples.

Answer 2

The easiest way to import data from a RDBMS is the DataImportHandler. Check out this step-by-step quick start.

Also, here's a pretty thorough walk-through of its usage.