First question you should ask yourself is: what do I need from this framework? JQuery and YUI, for example, take two totally different approaches. JQuery itself is pretty much a simple querying library that uses XPath selectors to navigate the DOM tree and apply transformations to its nodes. That being said, it also has tons of libraries (such as JQuery UI, as well as hundreds if not thousands of third party ones) that contain UI widgets, nifty animations, and other ridiculously cool things you never knew you could do so easily with Javascript. It's a functional library and doesn't really deal with objects.
YUI, on the other hand, goes the opposite direction. Primarily it's a UI library, meaning it exists to provide you with functional UI widgets (buttons, trees, panels, dialogs, etc). It also has additions and support/utility objects that can be used to do the same sort of XPath DOM tree querying as JQuery, but that's secondary. It's also pretty object-oriented.
The link cletus posted is also a great place to start.