Client-side partial page rendering and search engines

Question

Hi

I'm beginning to write what may grow to be a large commercial website. The business has several facets, so I'm considering a 'widget' based UI not dissimilar to the BBC homepage (http://www.bbc.co.uk).

I'm writing a content management system that would allow an administrator to compose pages using a selection of pre-defined widgets (i.e a text widget, a product widget, news headlines widget etc). I'm writing the application using ASP.NET MVC.

The mark-up for each widget type will be encapsulated in a user control (ascx). I'm considering two approaches for rendering the widgets (which I may mix):

1. Use RenderPartial to build the page up on the server (very much like the Kona sample from the ASP.NET MVC site)

2. Render widget place-holders on the served page then get the client to issue a request for each place-holder (using a jquery to call standard MVC actions returning HTML).

The second approach is appealing for two reasons:

1. It allows for a page to be returned without having to wait for slow-to-render widgets (like a comments widget, perhaps).

2. It allows me to control caching at a widget level (rather than at a page level). I can use the built-in OutputCache attributes to individually control the type of caching on each widget type.

Here's the question:

If some of the content of a page is only rendered by javascript-initiated HTTP GET requests (i.e. not included in the initial response), would this content be included in Google's appraisal/indexing of the page?

Assume that the javascript requests are triggered on document.ready

For those who like code, my prototype implements the second approach using this code at the server:

    [OutputCache(Duration = 30, VaryByParam = "widgetUniqueKey")]
    public ActionResult RenderWidget(int widgetId)
    {
        var cmsService = new CmsService();
        var widget = cmsService.GetWidget(widgetId);

        string viewName = widget.Accept(new ViewNameWidgetVisitor());
        object viewData = widget.Accept(new ViewDataWidgetVisitor());

        return View(viewName, viewData);
    }

with this code on the client (the initial page GET has returned placeholder divs)

    $('.widgetPlaceholder').each(function() { renderWidget(this) });

    function renderWidget(source) {
        var placeholder = $(source);
        var widgetId = placeholder.attr('id').replace("widget", "");
        var widgetUniqueKey = placeholder.children('div.uniqueId').text();

        $.get('/home/RenderWidget', {
            widgetId: widgetId,
            widgetUniqueKey: widgetUniqueKey
        },
                   function(data) {
                       placeholder.replaceWith(data);
                   }, "html"
               );
    }

The widgetUniqueKey will vary between widget types: it may be the product id for a product widget; the user id for a recommendations widget; etc.

Many thanks

Answer 1

No. The google bot (and any other indexers as far as I'm aware) doesn't execute any javascript code, so anything that you fetch via ajax won't be indexed.