Ferret the ruby implementation of lucene is reasonably powerful, however online discussions in 2008 seemed to indicate ferret had many stability issues and would segfault regularly. There have been 10 or so commits this year so the project has pretty light activity.
Is Ferret stable enough to use in production?
I used it for one project half year ago (July 2009). It was a database for one of festivals, so it just run for about 10 days (about 20 queries per minute with 50 updates per hour) and I had some problems. Few times I had problem with indexes and I had to rebuild it and few times server crashed. I didn't have time then to switch to something else, so I just added simple cron script that checked every minute if ferret server was running and, if not, it started it.
But I don't know how is it working now (I don't even know if there is a newer version).
Now I'm considering switching to something different, but I'll look into this later.
It seems that the community has pulled back from Ferret and the two primary contenders are Sphinx and Apache Solr.
While I do not have any hard evidence of "the community pulling back" (yes, its subjective) it just seems like there is not much inertia behind it and I think there are more feature-rich and mature options (again, Sphinx and Solr to just name a few).
It depends on your need. I've been running Ferret for 3 years now, and the past few months have a fairly complex Ferret deployment. I don't have crashes, ever, on production, but you have to be careful with your deployment. E.g. you have to absolutely make sure that you don't have multiple writers, but that's not difficult. If you want to customize with your own filter and analyzers, you can, but you have to test and make sure first you don't run into weird problems (I just ran into one and I think fixed it). The point is, if you are careful, you can get a good deployment going, no problem.
Ferret allows you to be very flexible and customizable in managing documents in your index. You can incrementally delete and update documents and fields, which is harder to do in Sphinx. You can also very easily assign weights to different fields. You can easily control how words should be indexed and searched. I think if you want to be flexible at building your new app, and want to try different ways to index words and weigh fields, Ferret's easy of use is a win.
I've never used Sphinx. I heard a lot of good things about it and it's actively developed (unlike Ferret). But my app requires very fine grained and frequent incremental updates, so I am stuck with Ferret.