With some tweaking (enable JQuery noConflict-Mode, replace the $ selector by something like $jq) you should be able to get to work both.
However you might easily end up with a Frankenstein app that mixes MooTools and Jquery plugins. Also the javascript load is quite enormous, because you have to include Mootools, JQuery AND their top-level UI toolkits.
I'd recommend that you investigate if JQuery UI could replace MochaUI (or vice versa). One combination of JS lib + UI toolkit is far easier to maintain.
The downside of MochaUI is that it has some issues with IE8, furthermore it is not very up-to-date (last release Sept 2008).
JQuery UI is much more modular (you can just pick the parts you need), has a visual CSS Theme Editor (Themeroller) and offers a 2-month release schedule. Its disadvantages: it doesn't yet cover essential UI components (such as a menu, a "viewport" manager that renders you UI components, palettes). However, these features are expected soon.