Is it common for robots to crawl inside iframes? And how do they do that? Do they index it instantly or do they just 'remember' the URL and continue sometimes later?
+1
A:
I would imagine so; I know googlebot does.
If in doubt, you can control things with robots.txt and a few extra links in your page footer. Look at the links down at the bottom of this page, for example.
tylerl
2009-09-21 08:02:43
That last bit makes no sense. What are you talking about?
random
2009-09-21 08:08:44
If part of a site is linked only via iframe, you can add an additional link someplace like the footer of the page. On the other hand, if you don't want a link to be indexed, then use a robots.txt file to block it out.
tylerl
2009-09-22 18:02:24
A:
An iframe just contains an HTML content like any other HTML page.
So yes engine bots will crawl them.
And you probably want them to crawl them as that's probably where your content is.
However you probably don't want users to access your website directly in the frame.
So you might want to detect the absence of frame. So you'll use javascript to add them they don't exist.
<script type="text/javascript">
if (top.frames.length==’0′) {
window.location.replace("Appropriate url with frames");
}
</script>
As a last comment I'd recommend you not to use frames. They're highly not recommended as they're absolutely against any accessibility policy.
They makes your website not having any content seen in their main page. So it clearly does not help your ranking.
If you can avoid them, do so (and you always can).
Damien MATHIEU
2009-09-21 08:25:43
I don't think this is true. Iframes don't contain any content (HTML). They are just like links for robots. You may be mistaken a bit, you are talking about frames, I am talking about iframes, but they are mostly same. Anyways, we are not using frames for layout, it is a bit more complicated and we have good reasons to use them, believe me :-)
Josef Sábl
2009-09-30 09:07:56
+2
A:
If there is an href/src link somewhere on an indexed (i.e. crawled) HTML/XHTML page, it will get indexed. Whether or not the page is presented to the user via an iFrame is irrelevant.
jldupont
2009-09-21 08:34:06
+1
A:
Iframes are easily crawled as others have pointed out. However I wanted to clarify that your page will not get "credit" for the iframed contents. So your page will not actually get indexed as containing the content -- it is merely a stepping stone for the crawler to index the iframed page and link directly to it. This is important to understand if the iframe points to content that is not in your domain, and you are expecting to leverage the iframe contents for your own page's visibility. Not sure from your question whether or not that is your intent.
bmoeskau
2009-09-21 09:03:46
Yes, I know, but good point. SEO is actually not important at the moment. We are comparing statistics of outer and inner page and we are wondering whether robots are to blame.
Josef Sábl
2009-09-30 09:03:31