A few months ago I tested the user population on a mainstream million-member site I was working on, and around 10% of unique users did not have Javascript running.
Consider reversing the question: is it worth developing a site that only works for Ajax-capable users? Would you really ignore search bots, most mobiles, and a heap of other users?
Back to basics. First, create your site using bare-bones (X)HTML, on REST-like principles (at least to the extent of requiring POST requests for state changes). Simple semantic markup, and forget about CSS and Javascript.
Step one is to get that right, and have your entire site (or as much of it as makes sense) working nicely this way for search bots and Lynx-like user agents.
Then add a visual layer: CSS/graphics/media for visual polish, but don't significantly change your original (X)HTML markup; allow the original text-only site to stay intact and functioning. Keep your markup clean!
Third is to add a behavioural layer: Javascript (Ajax). Offer things that make the experience faster, smoother, nicer for users/browsers with Ajax-capable JS... but only those users. Users without Javascript are still welcome; and so are search bots, the visually impaired, many mobiles, etc.
This is called progressive enhancement in web design circles. Do it this way and your site works, in some reasonable form, for everyone.