Automated testing for OpenGL application

Question

I have a Java Application that uses JOGL to provide a large part of the GUI.

Is there any tool which you know of, or have used which can automate the testing of OpenGL applications (or more specificly those using JOGL)

Just to update: The tool can run on either linux or windows.

Answer 1

I have written unit-tests for C++ (Qt on Linux) & OpenGL before. I don't know any reason it shouldn't work for Java too.

The things which worked for me are:

• Abstract your OpenGL context provider so the rest of your code is independent of it. In my case the main app used Qt's QGLWidget, but the unittests used a pbuffer-based one which I could create with no windowing infrastructure at all (other than a designated X11 DISPLAY). Later I added an "offscreen Mesa" (pure software OpenGL implementation) so they'd even work on a headless build machine with no GPU at all.

• Keep your OpenGL code independent of your GUI code. In my case the OpenGL "rendering engine" didn't know anything about Qt classes (e.g mouse events). Define your own testable API which isn't tied to any specific GUI concepts, and write tests for it.

• In the unittests, read the contents back from the framebuffer using glReadPixels and either hit them with some assertions about which pixels should be particular values, or go down the regression-testing route and compare the framebuffer capture with a stored image you know is good (either from manual verification, or because it's produced from some other reference model).

• Allow a bit of fuzziness in any image regression testing; most OpenGL implementations produce slightly different output.

• (I didn't do this, but...) Ideally you want to be able to test the GUI layer by asserting that it makes the expected sequence of calls to your rendering engine in response to GUI activity. If it does that, and you're confident of your render layer because of the above tests... well, no need to actually do the rendering. So create a suitable mock object of your rendering layer for use when GUI testing (I'd imagine tests like "mouse drag from here to there results in a call to the rendering layer to set a particular transform matrix"... stuff like that).

Answer 2

I've been thinking of using a picture-diff tool such as PDiff to test OpenGL code, by taking snapshots, saving them to disk and comparing with previous regression output. That way, really bad stuff (missing textures) pop up but humanly unnoticable things (such as the above mentioned small diffs between implementation) go through fine.

Also, for automating user interaction, either the GUI classes should be sufficiently open for you to send events or call 'clicked' on a button, or you have to inject OS events manually to your apps. This is possible, but much more troublesome. Might be easier to open up the GUI layer, if it's Open Source.