Some time ago I came across as site online who's sole purpose was the collection of various data sets, location data, district census data, or whatever sets community members were interested in maintaining.
My question is, do you know the site that I'm thinking of, or can you suggest any other sites that perform a similar service?
I'll suggest GeoNames, a great source for zip/postal, lat/long and lots of other geographical infomation.
Edit: I don't mean search generally, I mean to say: Amazon is hosting a few selected public domain data sets for all to use. You can find out what data sets they have available by searching.
What about sports: The baseball archive has stats about everything baseball from 1871... http://baseball1.com/statistics - great source of date if one wants to train some statistics.
Another that was just shown to me is sig.ma, a search tool for retrieving ontologies. If you're building a 'web 2.0', this would be a great tool for bootstrapping.