Are you using cvsnt? If you do, cvs2svn doesn't understand that your file is binary, because cvs and cvsnt flag binary files differently. It is simple enough to write a program that converts this. I had to do that.
Now, if you have a binary file that isn't marked as binary in cvs, or it is marked using the cvsnt syntax, and the file contains patterns like "$Id" or "$Date", then Subversion will substitute those patterns when you check out the files. Usually that means your binary file will get corrupted.
How to convert: If the file contains "kopt b;" assume it is binary, although technically a file may be binary in some revisions and not in others. For all binary files, insert "expand @b@;" before "symbols" near the top of the file, so the header looks something like this:
head 1.1;
access;
expand @b@;
symbols
When you do this, be careful not to change anything in the rest of the file, e.g. line endings.