Open Source Projects for Web Search Engine Components

Question

Can someone put a list of open source software of each of these components included in the question "Building a web search engine"?

That will save many people's time in the search for high quality open source components.

Component List:

1. Crawler
2. Parser
3. Indexer
4. Ranking Engine
5. Front End

Answer 1

For the Java world:

Parsers

• JavaCC
• Antlr

Indexer

• Apache Lucene

Ranking Engine

• Apache Lucene

Front End

• whatever lightweight MVC framework

Answer 2

Lucene is also available for .NET http://incubator.apache.org/lucene.net/

Answer 3

It looks like nutch does everything you need out of the box.

Answer 4

Nutch + Lucene + Mapreduce/Apache is your solution.