views:

1034

answers:

3

Given the recent 55 minute Google service interruption, it seems obvious to me that internet search is too important to leave in the hands of one company (no matter how hard they try not to be evil). So are there some (possibly p2p) free open source alternative internet search/information storage and retrieval systems that might work or be improved enough to take Google's place?

One specific idea: there is a lot of information about Google's technology already out there (MapReduce, PageRank). Would it be feasible to use open source implementations of those technologies to create some kind of free (p2p?) version of Google search?

Also, is there any chance now with the current state of semantic web technologies that some of them could be promoted to larger scale use and make services like Google unnecessary?

+1  A: 

Even if everything Google created was open source, you still would have to offer the massive network infrastructure that they do, so I'm skeptical about Google becoming irrelevant or unnecessary.

Otávio Décio
So if everyone thinks Google is so necessary and completely irreplaceable, why are we treating them like an ordinary company? Maybe they should have their own UN envoy or something.
Jason Livesay
Maybe you can come up with something better and win their market.
Otávio Décio
They're not so necessary nor are they completely irreplaceable. If they fuck up enough, search market share goes to Yahoo, MSN, or a new upstart. Google was once an underdog taking marketshare from Altavvista, remember.
ceejayoz
+6  A: 

While the software may be available (like Hypertable), you'll still be hard pressed to match the scale of hardware available in Google datacenters. Also consider the fact that Google is now reportedly designing it's own routers to cope with the demands of network traffic on their own networks. These are not simple problems and it's taken them 10 years to get to where they are now.

The most likely scenario I could see is developing a similar system using open source tools, and running it on an open cloud computing platform (like Amazon EC2). You'll still be at the mercy of hardware provider, but you'll be free to run the software as you see fit.

Well, there's also the small matter of how to fund such an operation.

In short, it can be done, but it's very unlikely to become a viable alternative.

Regarding P2P: it looks nice on paper, and BitTorrent has certainly proven that it can work, but there are major real world problems yet to solve to do something like distributed search on the entire WWW. I'm not familiar with research in this area, but all the attempts at p2p storage/retreival so far have not worked that well.

Mark Renouf
Well, seems like people aren't considering the p2p idea I put in the question. There is no way that some kind of p2p software could be distributed that would have Hypertable as part of it that just about everyone would have, like most people have BitTorrent?
Jason Livesay
The major problem of p2p search is latency. People have tried and failed (with interest findings) However it's conceivable that in the future when most people have their cloud instances in data centers, an open source, anonymous and totally democratic search engine will rise to slay the dinosaurs :)
ididak
+6  A: 

The last time i remember google being down was on may 7th of 2005 when it had DNS issues. That was 15 minutes.

So 70 minutes of downtime for the last 1365 days. Or 0.00003% downtime.

Ólafur Waage
Plus, today's wasn't downtime. Search results could still be accessed by removing a portion of the URL.
ceejayoz