SEO: Duplicated URLs with and without dash "/" and ASP.NET MVC

Question

Hello guys, after reading this article "Slash or not to slash" (link: http://googlewebmastercentral.blogspot.com/2010/04/to-slash-or-not-to-slash.html) on Google Webmaster Central Blog (the oficial one) I decided to test my ASP.NET MVC app.

For example: http://domain.com/products and http://domain.com/products/ (with "/" in the end), return the code 200, which means: Google understands it as two different links and likely to be a "duplicated content". They suggest to choose the way you want... with or without dash and create a 301 permanent redirect to the preferred way.

So if I choose without dash, when I try to access http://domain.com/products/ it will return a 301 to the link without dash: http://domain.com/products.

The question is, how can I do that with ASP.NET MVC?

Thanks, Gui

Answer 1

you need to check the URI in the INIT event and check the URI to see if it coming in with the slash, if it is, simply do a redirect and add the 301 header to the output response.

Answer 2

If your using IIS 7 you could use the URL Rewrite Extension ScottGu has a blog post about it here.

Alternatively if you want to do it in code you could inherit from PerRequestTask. Here some sample code the removes the www from an address - this is from Shrinkr:

public class RemoveWww : PerRequestTask
{
    protected override TaskContinuation ExecuteCore(PerRequestExecutionContext executionContext)
    {
        const string Prefix = "http://www.";

        Check.Argument.IsNotNull(executionContext, "executionContext");

        HttpContextBase httpContext = executionContext.HttpContext;

        string url = httpContext.Request.Url.ToString();

        bool startsWith3W = url.StartsWith(Prefix, StringComparison.OrdinalIgnoreCase);
        bool shouldContinue = true;

        if (startsWith3W)
        {
            string newUrl = "http://" + url.Substring(Prefix.Length);

            HttpResponseBase response = httpContext.Response;

            response.StatusCode = (int) HttpStatusCode.MovedPermanently;
            response.Status = "301 Moved Permanently";
            response.RedirectLocation = newUrl;
            response.SuppressContent = true;
            response.End();
            shouldContinue = false;
        }

        return shouldContinue ? TaskContinuation.Continue : TaskContinuation.Break;
    }
}

You would just need to check for the url ending with a / in your code.

** Note this does use a 3rd party dll - System.Web.MVC.Extensibility namespace. **

Answer 3

It dosnt matter really for Google, but what does matter is if both urls' http://domain.com/products and http://domain.com/products/ show the same page, you also need to watch with windows servers that links to your site like from external pages where the user has typed http://domain.com/PRODUCTS/ will aloso be seen as a diffrent page as the web is case sensitive.

There is away round this with the use of canonical url meta tag, it tell s google what the page name is really, so will avoid duplicate pages which ant really diuplicate

http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html