I will answer this myself since I've done the research. Here's a summary:
HTML/Ajax, Flash, and Flex are
different approaches to developing
web apps. These technologies can be
combined in a single web app.
Adobe AIR is a runtime which allows you to take those web technologies and run them as a desktop app instead of in the browser. You can combine them in the same app.
Flash is a runtime that was originally created to provide a rich UI to browser based apps. AIR brings the Flash runtime to desktop apps.
Flash runs compiled SWF files and is scripted via ActionScript. Flex is a tool
(SDK, MXML schema for defining UI, etc), which makes it easier to create apps that run on the Flash runtime (it generates SWF files).
Further details see: Adobe AIR FAQ, Adobe AIR (Wikipedia), Adobe Flash (Wikipedia), Adobe Flex (Wikipedia).
As for deciding which approach(es) should be used when developing for AIR, I think that should be based on the experience and preferences of the developers working on it. I don't think Adobe gives any guidance here, because their goal is simply to provide the same options for desktop developers that web developers already have.