I want to learn SEO！ How much does it help to know how robots work when doing SEO? So could anybody kindly answer that? Maybe it is a dumb question. Thanks!
+3
A:
Assuming that by robot you mean a web spider AKA crawlers, the details of how it works are not going to be particularly useful to do search-engine optimization -- it follows links, essentially, not much more that's relevant to your purpose!
Of course you need to understand the de facto standards: writing robots.txt files to exclude crawlers from some pages, writing good sitemaps, how to increase your crawl rate, use webmaster tools, landing page optimization, and so on -- all skills pretty different from understanding the inner workings of spiders.
Alex Martelli
2009-06-11 03:31:17
Thank you very much! it seems I made a mistake. I thought "writing good sitemaps, how to increase your crawl rate, use webmaster tools, landing page optimization, and so on" are highly relevant to understanding the inner workings of spiders. It seems absolutely worthwhile for me to lose my reputation to learn this. Your answer saves me lot of time, I do not know how can I reward you with my poor reputation, though. Thanks!
tag
2009-06-11 08:04:12
Maybe just a matter of wording: to me, for example, landing page optimization is almost entirely about understanding the behavior of _human_ visitors (optimizing their user-experience, and thus your conversion rates, site-stickiness, etc) rather than spiders.
Alex Martelli
2009-06-11 14:45:33
+2
A:
Yes and no. You need to understand things some things:
Note that I'm assuming you mean a standard web crawler, not youtube or something like that. And Pagerank assumes you're focusing on Google, which is generally a good idea. ;)
But, you shouldn't care about the code; there's too much noise. The algorithms that go behind stemming aren't relevant to SEO. How the crawler deals with timeouts isn't super useful. And you can ignore the details of how to fetch web pages in bulk -- high-scale parallel data processing can be a thorny topic, and you just don't need to know all that.
Check out SEO-oriented blogs, tools, and websites. Or you could just search for SEO; if the best answers aren't right there, then they're doing something wrong. ;)
ojrac
2009-06-11 03:44:10
A:
I would suggest that you take a look at SEOmoz's search engine ranking factors page. It is an organized survey of seo professionals' opinion regarding 50 or so factors. It is very informative and can get you up to speed with what is considered relevant, relatively quickly.
simplemotives
2009-06-15 21:07:03