Two possible downsides are that it could make it difficult for users to bookmark content on your site and difficult for search engines to differentiate pages on your site.
You should probably provide more information on your reasons for taking this approach. You might have good reasons or it might be a case of using a technology (AJAX) because it is cool to use.
If you want to give the users the impression of fast responsiveness, then yes AJAX load your pages, but still have a different url for each page. This will take more code but it will solve both issues that I mentioned.
http://yourdomain.com/home.aspx //loads its own content via AJAX
http://yourdomain.com/contact.aspx //loads its own content via AJAX
etc
This is really only appropriate if you have a lot of content, or where the content involves time-consuming calculations, such as on a financial site. In most cases, it would be less trouble to just load your pages normally or break you content into paged chunks.