In my opinion, images which are part of content should use image tags, and everything else can use background images/CSS sprites.
By including a normal image tag with an alt attribute, you will make the image available for things like google image search. But, this is not a factor I weigh very heavily.
The important part is that if someone looks at your site with a screen reader or simply views the HTML, the content will be successfully conveyed with an image tag and alt attribute. If you use divs with CSS you will lose this effect, and some people might miss the content.
But, when it comes to non repeating background images, buttons, and other style oriented images, I do recommend using CSS Sprites to improve performance.
While putting the other images in real image tags should preform slightly worse, this can be very minimal if you set the caching headers properly on the images.
Edit:
A quick search found this:
http://www.google.com/support/forum/p/Webmasters/thread?tid=75fa18ce5e671f5b&hl=en
Which seems to mitigate the Alt attribute by placing text in the div and then using sprites to hide the text:
The approach Google uses, relevant text inside link, visible for crawlers, people who have images turned off, screen readers, etc.
<a href="..." style="position:relative; display:block; overflow:hidden; width:100px; height:100px">
relevant text here
<span style="position:absolute; top:0px; left:0px; width:100px; height:100px; background-image:url('sprite.jpg'); background-position:0px; -100px;"></span>
</a>
If performance is likely to be an issue it might be worth going that route but I do prefer the more semantic img tags if they are part of the content.