Where can I download a free, text-rich dataset?

Question

I want to do a bit of lightweight testing and bench-marking for full-text search, so the dataset should have the qualities:

• 10,000 - 100,000 records.
• good dispersion of English words.
• In CSV or Excel format--i.e. I don't want to access it via API.

Something like books or movies with title and description fields would be perfect. I browsed the UCI Machine Learning Repo, but it was too number-oriented. Thanks!

Answer 1

If you do not find one, you can create one using the LOREM IPSUM generator

http://stackoverflow.com/questions/1126725/t-sql-equivalent-of-rand

You can also get the full StackOverflow data dump

http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/

Answer 2

You can try
- CKAN
- or search for "Open Data"

Or, see Tim Berners-Lee discussing a quick survey of a few Open Data sets.

Answer 3

Use the Gutenberg Project. You have access to thousands of English book in plain text. That's what I've used once and was happy with it.