When a user enters my page, I have to make another AJAX call...to load a data inside a div. That's just how my application works.
The problem is...when I view the source of this code, it does not contain the source of that AJAX. Of course, when I do wget URL ...it also does not show the AJAX HTML. Makes sense.
But what about Google? Will Google be able to crawl the content, as if it's a browser? How do I allow Google to crawl my page just like a user would see it?
GoogleBot doesn't interpret Javascript. Make sure you present whatever information you want crawled through HTML/XHTML on the page.
Also, make sure you don't present a different page to the crawler (based on the UserAgent field) because this could get you in trouble, SEO wise.
Web crawlers have a difficult time with ajax and javascript that dynamically loads content. This site has some ideas that show you how to help google index your site http://www.softwaredeveloper.com/features/google-ajax-play-nice-061907/
If you make your pages such that they will work with OR without javascript (i.e. fall back to using frames or standard GET / POST requests to the server if javascript fails, either automatically, or via a "display as plain html" link ), it will be much easier for search engines to crawl the page.
It makes sense for them not to crawl "dynamic" content - because it is just that...dynamic.
My understanding is that in most situations, Google does not crawl the client-side-dynamic-content.
GoogleBot does not interpret javascript.
If you want to see your webpage as the GoogleBot would, I would recommend looking at a text only browser such as lynx.
Don't rely too heavily on javascript and AJAX for your content if you want google to be able to properly index it.
What I do in this situation is always initially populate the page with content based upon the default parameters of whatever the Ajax call is doing. Then I only use the ajax javascript to do updates to the page.
As other answers say, Google's crawler (and I believe those of other search engines) does not interpret Javascript -- and you should not try to differentiate by user-agent or the like (at the risk of having your site downgraded or blocked for presenting different contents to users vs robots). Rather, do offer some (perhaps minimal) level of content to visitors that have Javascript blocked for whatever reason (including the cases where the reason is "being robots";-) -- after all, that's the very reason the noscript tag exists... to make it very, very easy to offer such "minimal level of content" (or, more than minimal, if you so choose;-) to non-users of Javascript!
Actually... Google does have a solution for crawling Ajax applications...
http://code.google.com/web/ajaxcrawling/docs/getting-started.html