views:

122

answers:

2

How does one test if an application rendered something correctly?

For example (2D example):

Microsoft Word 2007

How does one know that the shadow is placed correctly or the correct color / outline was rendered? Or if 3D effect renders correctly when one would rotate in a direction? Other things could be if the word art was re-sized, how does one measure its 'correctness'?

+3  A: 

There are a few ways:

  1. If it's actually very precisely specified what should be rendered and in exactly which way, then you can just compare the pixels to a reference rendering.
  2. In case of things like SVG it's not so clearly specified. The usual approach here is to use reference renderings and compare them by hand. You can easily overlay both, subtract one from the other and spot glaring differences that way. It's not an automatic process, though.
  3. You can look at the data representing the drawn image instead of the image on screen directly. What gets drawn is (in your example) a vector graphic. That means there are several shapes which should have well-defined properties, shapes and colors and you can simply compare your shape data to a reference. That kind of things can be done automatically. I think Google uses a similar approach to compare Chrome's rendering with reference renderings of web pages; they don't compare pixel data, they compare the higher-level data what and how the browser would render.
Joey
In the 3rd testing method, are you checking it against what would be perceived as a correct render? Doesn't that add an element of subjectivity? Also what if there is no render you can refer to, will the only choice be method 3?
srand
The 3rd one actually looks at the data structures that will be used to render whatever will be seen. So you're just taking the comparison to another abstraction layer. You would compare that the shadow shape actually is located at the right position, etc. Surely, you *need* to know what would be "correct". But otherwise you'd have a hard time classifying an image as correct or incorrect output, anyway. And yes, reference renderings are in fact somertimes drawn by hand. They have to be accurate to the specification you're testing.
Joey
+2  A: 

There are two ways: The image and the rendering based one.

The image way: You must find a way to render the image to an internal pixel buffer (so you can run your tests "headless", i.e. without an actual UI popping up).

Then select a few pixels and make sure their colors are correct. In your example, select a couple of white pixels around the text to make sure the rendering doesn't leak. Helper methods like assertThatRectangleIs(area, color) will help cover some ground. The idea is to select specific areas (like the left vertical bar of the H) and not be too picky.

The rendering based way works with the assumption that your gfx library works. So what you do is you mock the actual rendering code with something like this:

public class MockGC extends GC {
    List<Op> ops = new ArrayList<Op> ();
    void drawLine (int x1, int y1, int x2, int y2) {
        ops.add(new Line(x1, y1, x2, y2, copyGC (gc)));
    }
}

so you just save the command and all relevant options in a data structure that is easy to compare. Or even:

public class MockGC extends GC {
    StringBuilder buffer = new StringBuilder ();

    void drawLine (int x1, int y1, int x2, int y2) {
        buffer.append("line "+x1+","+y1+" - "+x2+","+y2+", color="+foreground()+"\n");
    }
}

Later, you can then just verify that the correct rendering commands have been issued and that the GC (colors, font, rendering hints) was used.

The latter way is much faster and 100% precise but much more work to code.

Aaron Digulla
Can you expand on your 2nd method?
srand
What you want to know? It's pretty simple: Just save the command and all relevant options in a structure that's easy to check.
Aaron Digulla