Depends what you wish to do with your data.
If you only want to display votes than I say you only use one field. It's like number of views of a thread on forum. You want to see what gets most clicks, but not how many times someone viewed it.
Voting system on SO is a bit more complex. Since they can cancel all votes from particular user they have to keep track of who voted for/against what. This, I think, is written in another table, but because it is expensive to recalculate all votes every time someone views a question, they keep calculated value in a field, changing it whenever someone votes.