Yahoo offers OpenID buttons on this webpage: http://developer.yahoo.com/openid/loginbuttons.html
I was wondering if Google / Aol and other providers have similar graphics that I can use on my webpage. If so, where can I find them? Otherwise, am I alllowed to just photoshop the logos of these companies into my own graphic?
Technically, you need permission to use the branding of any company. Yahoo! just happens to be nice and provide shiny buttons. Of course, I doubt anyone is going to come hunt you down, unless you are damaging their brand somehow. Consult a lawyer if you are concerned... I am not one.
The openid-selector has some icons that you may be able to use. The project is distributed under the New BSD License.
Take a look at their demo page to see the icons.