For data modeling, it is usually important to know what things mean.
Are an up vote AND a down vote on the same thing by the same person are the same as no vote at all by a person on a thing?
In any case, it's certainly possible (with appropriate indexing for performance) to have it in a single table for ease of management. On top of that base table you could add UpVote and DownVote views. Or you could go the other way and have two base tables with a single Votes view for convenience.
Typically, having two tables now means two sets of referential integrity, two semi-identical table schemas to be maintained, and a not-quite unified view of the data. It also means you might have to have two archive and/or partitioning plans for handling history, etc.
That doesn't mean it's wrong, but if entities are very similar, modeling them in a single table often makes a lot of sense.
Sometimes entities may appear similar, but in fact, they really aren't and they should be be modeled in separate tables. Quotes, Purchase Orders and Invoices come to mind here. A lot of people like to shoehorn two or more of these things into the same table, and that's really a bad idea.