Since this is a fairly subjective question I will put in my 2 cents.
I've developed using the Dojo toolkit and found it very well rounded. However, if you don't do a custom build on your library it slows everything down. It has a steeper learning curve than other frameworks, but seems to sit pretty close to heart of Javascript.
But this isn't an advertisement for dojo. I'm working on a Ajax web app currently and my goal is to try and avoid using a framework. Why? A few reasons.
The biggest reason is complete control over the code. Not that I don't trust toolkits, I just like my custom code. I don't have any code that I don't use, and all of my abstractions are custom made for my purposes. At times I am in fact reinventing the wheel*, but its always a bit different because it's purpose built.
The other big reason is the amount of knowledge I gain. I'm confident that I could pick up any framework much faster, because of not using one for a project. I understand so much more than I did previously. Before It was more like a voodoo mystery box, and now even frameworks make more sense to me.
The fact of the matter is if you do any non-trivial javascript you will end up with a framework (or a mess). In reality it may not be the fastest way to write web apps, but it isn't all that hard with a little practice. You can be amazed at what creative things you can come up with that didn't exist before.
*Reinventing the wheel is a terrible analogy that we all use. We reinvent wheels all the time. First made of stone, then lighter wood, tires, etc... My truck doesn't have the same wheels as my sedan.