Free open source alternative to Google?

Question

Given the recent 55 minute Google service interruption, it seems obvious to me that internet search is too important to leave in the hands of one company (no matter how hard they try not to be evil). So are there some (possibly p2p) free open source alternative internet search/information storage and retrieval systems that might work or be improved enough to take Google's place?

One specific idea: there is a lot of information about Google's technology already out there (MapReduce, PageRank). Would it be feasible to use open source implementations of those technologies to create some kind of free (p2p?) version of Google search?

Also, is there any chance now with the current state of semantic web technologies that some of them could be promoted to larger scale use and make services like Google unnecessary?

Answer 1

Even if everything Google created was open source, you still would have to offer the massive network infrastructure that they do, so I'm skeptical about Google becoming irrelevant or unnecessary.

Answer 2

While the software may be available (like Hypertable), you'll still be hard pressed to match the scale of hardware available in Google datacenters. Also consider the fact that Google is now reportedly designing it's own routers to cope with the demands of network traffic on their own networks. These are not simple problems and it's taken them 10 years to get to where they are now.

The most likely scenario I could see is developing a similar system using open source tools, and running it on an open cloud computing platform (like Amazon EC2). You'll still be at the mercy of hardware provider, but you'll be free to run the software as you see fit.

Well, there's also the small matter of how to fund such an operation.

In short, it can be done, but it's very unlikely to become a viable alternative.

Regarding P2P: it looks nice on paper, and BitTorrent has certainly proven that it can work, but there are major real world problems yet to solve to do something like distributed search on the entire WWW. I'm not familiar with research in this area, but all the attempts at p2p storage/retreival so far have not worked that well.

Answer 3

The last time i remember google being down was on may 7th of 2005 when it had DNS issues. That was 15 minutes.

So 70 minutes of downtime for the last 1365 days. Or 0.00003% downtime.