Webapps: Storing and searching through user submitted blocks of text.

Question

Background:

I'm building a poetry site with user submitted content. The relevant user actions for my questions are that users can:

• a. Go to fancysitename.com/view to see all poems so far
• b. Go to fancysitename.com/submit to submit your own poem.
• c. Go to fancysitename.com/apoemid to view a particular poem you've bookmarked before.
• d. Go to fancysitename.com/search to enter a word to search for in all the poems.

All the poems are stored as text fields in a database and referenced by a poem id. So the "apoemid" in step c will be the primary key of the tuple and I'll just pull up the text after getting the key from the url.

Question:

• 1. The poems exist nowhere except in a database. My webapp is literally 4 html files. Will this approach affect my search engine rankings?
  2. Is there a more efficient way to do 'd' rather than do a Select * on the db and manually parsing the text on the server? Each poem will be at the most 10 lines long, so I would imagine using a full text search engine like Lucerne will probably be overkill.

Caveat

I'm running this on the google app engine for now, so my database customization options are pretty limited. So while I'd certainly be interested in hearing about the ideal way to do this, this is a pet side project so my budget is limited :(

Thanks!

Edit: Apparently I don't google so well at 7am. I've since found a solution for question 2 here so please disregard question 2.

Answer 1

In most database engines, you can accomplish this kind of searching. For example MysQL does have full text searching. I am not sure how app engine works but you can always have a stored procedure does this search.

Answer 2

AppEngine currently doesnt support full text indexing, they do have a better than nothing SearchableModel.

Some details of SearchableModel can be found here:

http://groups.google.com/group/google-appengine/browse_thread/thread/f64eacbd31629668/8dac5499bd58a6b7?lnk=gst&q=searchablemodel

Regarding search engine ranking, yes having all your poems in the datastore can affect your ranking. This is generally overcome through the use of a sitemap. Here is an article about how StackOverflow uses a sitemap to help its search ranking.

http://www.codinghorror.com/blog/archives/001174.html

Answer 3

Where you store your data will not affect your site's ranking, only how you serve it up (on what URLs, etc). There's absolutely no way for an arbitrary search spider to tell where you store your data, and no reason for it to care, either.

Regardless of the length of your text, you will need full-text searching if you want to search inside a string. As Sam points out, SearchableModel ought to work just fine for that.