This is a bit of a daunting task. The suggestion to live and let live both frameworks is not a bad one, provided things are working. It is likely jQuery is in noConflict
mode if they are really resident on the same page.
One tool that's very useful is to use the Web Developer Toolbar for Firefox and navigate to a page and do: Information -> View JavaScript
-- this will give you a full listing of all the JavaScript script loaded with <script>
tags, as well as inline JavaScript in <script>
tags, though I think inline event handlers are not listed there.
But one thing you can do is simply unhook Prototype or JavaScript and see which ends you up with fewer errors. Then you're on your way to debugging what's missing.
It is easier to standardize on one library, but there's a case to be made for "if it ain't broke, don't fix it" too.
I've taken some time to learn a bit about all frameworks, doing the same effects on identical HTML: ArtLung Rosetta: You can get a quick dive into the differences between syntaxes of Prototype and jQuery by comparing this and this.
Best of luck!