Yes, web accessibility makes the content more accessible for everyone and everythng, not just for handicapped people. You can imagine it's like installing a ramp next to the stairs -- instead of installig a specialized elevator for wheelchairs. With the ramp, everybody can use it -- people who walk just fine but feel like using it, people who need to carry something heavy, people with trolley carts, etc.
So, with your content made accessible, not just people browsing your web page will benefit -- also all kinds of automatic indexing programs, like the one that Google uses to index and rank pages, any scripts and plugins that your users will install in their browsers, and -- last but not least -- your content will be easier to access or recover in case of a failure -- for example, when your image-serving host dies and suddenly no images are loaded.
On the other hand, be careful about blindly following guidelines and using automated tools -- this is something that needs to be thought about, and at least tweaked manually. Unless you are legally obliged to follow the guidelines, you will want to pick the ones that give you the best improvements at the lowest cost, and that is very specific to your content.