What should i know about search engine crawling?

Question

I don't mean SEO things. What should i know. Such as

1. Do engines run javascript?
2. Do they use cookies?
3. Will cookies carry across crawl sessions (say cookies from today and a craw next week or month).
4. Are selected JS filters not loaded for any reason? (Such as suspected ad which is ignored for optimization reasons?)

I don't want to accidental have all index page say some kind of error or warning msg like please turn on your cookie, browser not supported, or not be indexed because i did something silly such as having my sitemap point to /r?id=5 and not have then index because it is a redirect (i would use 301 however).

Answer 1

From here: http://www.google.com/support/webmasters/bin/answer.py?answer=35769

Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as JavaScript, cookies, session IDs, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site.

Read Google's Webmaster guidelines