It's a matter of encapsulation. I put everything I need to render the view (including dropdown list data and server-side validation) into the ViewModel object. That way I don't have stuff scattered to the high winds.
I'm not a purist, but the idea of pushing untyped information into the ViewData, and casting it to the correct type in the view, just feels uncomfortable to me. I have yet to find a situation where this makes sense, unless I am passing small bits of information to the view that have nothing to do with the ViewModel or the underlying database model.
Small bits of information that I would put in the ViewData might include things like paging links, or perhaps ReturnUrl references that I don't want to put in the Url as a parameter, nor do I want them polluting my ViewData object.