One factor to consider is exactly how many votes you're going to be seeing. If we're talking on vast orders of magnitude (say more than tens of thousands of votes on each of hundreds of thousands of items), then storing every vote could end up using more resources than it is worth.
On the other hand, if there's a reasonably small number of votes (say less than a few hundred records for each of a few hundred items), then it might be advantageous to store each vote for the added flexibility it can provide in future development. A month from now, you might think of something new to do with this information, and having the backlog of votes could be very helpful in tailoring how that should work.
As others have mentioned, this could easily be combined with a meta table to sort things easier and simplify queries. Personally, I would recommend a view for this type of thing, as doing so will decrease your overhead by making updates automatic for the fields containing derived data.