Image processing and compression algorithms are typically compared on sets of widely available, well known images.
Is there a collection of video files for the same purpose, that are publicly available, or a set of raw video data that could be used for comparison?
So far, I have found data files of uncompressed video for the Dirac project.
The image here of Lena seems to be the standard one used in all image processing examples!!
Other than that I've not heard of a standard test suite, but here's the sort of images/videos we use to test compression (basically anything with lots of details/edges):
Aside from that make sure you test it with real world examples of the scenes that it will actually be used on!!! It's no good testing that the compression works well with static scenes of a building, but fails miserably when pointed at a tree blowing in the wind.
It really depends on what you're trying to test for. Recently, we've started looking for videos with faces, moving people, hands, and particular objects to detect certain pattern recognition software. Additionally, you really want to make sure you understand the code so you can unit test without relying on particular instances of video. However, it may not always be possible to unit test, so for black-box testing identify the main goals and then produce/find videos with those frames.
Additionally, since most video is just frames of images you could try and apply interesting distortion algorithms to standard images to create interesting videos. You may also want to take a look at blender to make some cool videos.
Main thing is really understand what you're testing for and then find test cases for that purpose. If you're having trouble, let us know what you want to test and we'll help you out!