Canvas App - facebook puts your code directly to the page (well, kind of), you have access to FBML (facebook specific tags), limited HTML, CSS (cached on Facebook side) and limited JS known as FBJS (a wrapper for native JS commands for security reasons, no jquery or anything like that). Your canvas app is still hosted on your server though, and you can use Facebook PHP API on a server side (and anything else you wish, it's a page on your server).
Frame App - facebook just puts an iframe on the page embedding some page on your server. Iframe could contain anything you want, it is just a regular page with no extra requirements. You can use HTML, JS (jquery etc), Facebook PHP API on server side. There is no FBJS as it is not needed, but you can use JS API (with similar functionality to PHP API, only for frontend). FBML is not directly supported, but you can use FBML tags on any site after declaring them:
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml">
Advantages of using Canvas App - no annoying iframe scrolling, quick access to FBML, and more close integration as a whole.