If you forget the alternative text on images, the site will be not-navigable for those people using text browsers, screen readers etc.
Search engines will of course find and follow any link they can find on a page. But it won't be easy for people, especially, if you're using SEO-unfriendly URLs with lots of parameters, so that you will never know what is there until you follow the link.
Also keep in mind that some people on dial-up connection (there are much more of those than you think) have a habit of switching off displaying images on pages (to save on traffic since they pay per megabyte). It's the same group who actively block ads in form of banners and especially Flash for the same purpose. Your images-only navigation may piss them off as well.