I have installed some error logging onto my ASP.NET MVC site and I have been getting a lot of NullReferenceException errors when google crawls my site.
But when I go to the URL that generated the error there is nothing wrong?
Why would this be happening? Why would the google bot be causing an error?
Something requiring cookies or a session or something else that Google doesn't support? Can you get any more information about the error from the event log?
Maybe missing cookie or referrer? Check for any dependency to optional HTTP headers.
Bots don't contain a whole lot of things that browsers do. For example you may be checking for a session variable that doesn't exist.