views:

3401

answers:

11

The calculations in my code are well-tested, but because there is so much GUI code, my overall code coverage is lower than I'd like. Are there any guidelines on unit-testing GUI code? Does it even make sense?

For example, there are graphs in my app. I haven't been able to figure out how to automate the testing of the graphs. It takes a human eye, AFAIK, to check if the graph is correct.

(I'm using Java Swing)

+12  A: 

Designs like MVP and MVC typically try to abstract as much logic out of the actual GUI as possible. One very popular article about this is "The Humble Dialog Box" by Michael Feathers. Personally I've had mixed experiences with trying to move logic out of the UI - sometimes it's worked very well, and at other times it's been more trouble than it's worth. It's somewhat outside my area of expertise though.

Jon Skeet
??? "sometimes it's worked very well, and at other times it's been more trouble than it's worth" Very weird because most new language tend to be MVC oriented... which is logic out of the gui (Model-Visual)...
Daok
+4  A: 

Here is some tips:

Try to remove the most code you can from the GUI (have controller, and model object) this way you will be able to test them without the GUI.

For the graphic, you should test the value that you supply to the code that generate the graphic.

Daok
+2  A: 

You can use JFCUnit to test your GUI, but graphics can be more challenging. I have on a couple of occasions taken snapshots of my GUI and automatically compared it to a previous version. While this doesn't provide an actual test, it does alert you if an automatic build fails to produce the expected output.

Steve Moyer
+3  A: 

You can try UISpec4J is an Open Source functional and/or unit testing library for Swing-based Java applications...

CMS
I've started using UISpec4J for a month or so for doing requirements validation testing on Java Swing GUI's. I've been liking it a lot so far.
Bassam
A: 

There is Selenium RC, which will automate testing a web based UI. It will record actions and replay them. You'll still need to walk through the interactions with your UI, so this will not help with coverage, but it can be used for automated builds.

David Robbins
A: 

From what I know, this is quite complicated, and really depends on the language - numerous languages have their own way of testing GUI, but if you really need to test the GUI (as opposed to model/gui interaction), you often have to simulate an actual user clicking buttons. For example, the SWT framework used in Eclipse provides SWTBot, JFCUnit has already been mentioned, Mozilla has their own way of simulating this in XUL (and from what I've read on their blogs, these tests seem to be quite brittle).

Sometimes you have to take a screenshot, and test for pixel-perfect rendering (I believe that Mozilla does this to check for correctly rendered pages) - this requires a lengthier setup, but might be what you need for the graphs. That way, when you update your code and a test breaks, you have to manually check the image if the failure was real, or you improved the graph rendering code to generate prettier graphs and need to update the screenshots.

Kim Sullivan
+6  A: 

Of course, the answer is to use MVC and move as much logic out of the GUI as possible.

That being said, I heard from a coworker a long time ago that when SGI was porting OpenGL to new hardware, they had a bunch of unit tests that would draw a set of primatives to the screen then compute an MD5 sum of the frame buffer. This value could then be compared to known good hash values to quickly determine if the API is per pixel accurate.

dicroce
I like this approach. Cumbersome to set up and maintain, but it would give me the regression testing capabilities I need.
Steve McLeod
Tried this. Failed miserably. See http://stackoverflow.com/questions/528224/how-to-use-linux-fonts-in-windows-qt
danatel
danatel you missed the point. OpenGL is a pixel accurate graphics rendering API. GUI application frameworks like Qt are going to be highly dependent on the system chrome, so hashing the frame buffer is not a good approach.
Bob Somers
An MD5? What has a cryptographic hash to do with this? How is security involved? A hash, OK, it's a great idea for pixel-perfect graphics but a cryptographic hash? You can use faster hashes that are non-cryptographic and whose probability of collision is negligible. Are you sure it was a cryptographic hash and not simply a hash? (in addition to that, in this day and age of parallel computing, hashing algorithm [used for non-cryptographic purposes] that cannot be parallelized should be looked upon with suspicion)
Webinator
A: 

If you are using Swing, FEST-Swing is useful for driving your GUI and testing assertions. It makes it pretty straightforward to test things like "if I click button A, dialog B should be displayed" or "if I select option 2 from the drop-down, all the checkboxes should become deselected".

The graph scenario that you mention is not so easy to test. It's pretty easy to get code coverage for GUI components just by creating and displaying them (and perhaps driving them with FEST). However, making meaningful assertions is the hard part (and code coverage without meaningful assertions is an exercise in self-deception). How do you test that the graph was not drawn upside-down, or too small?

I think that you just have to accept that some aspects of GUIs cannot be tested effectively by automated unit tests and that you will have to test them in other ways.

Dan Dyer
+1  A: 

What I gather from your question is that you're looking for an automated way to test your GUI behavior in detail, the example you give is testing whether a curve is actually drawn correctly.

Unit testing frameworks provide a way to do automated testing, but I think the kind of tests you want to do are complicated integration tests that verify the correct behavior of a multitude of classes, among which the classes of your GUI toolkit/library, which you should not want to test.

Your options depend very much on what platforms/toolkits/frameworks you use: for example, an application using Qt as its GUI framework can use Squish to automate its tests. You verify the results of your tests once, and subsequent automatically executed tests compare results against the verified results.

andreas buykx
+1  A: 

You can try to use Cucumber and Swinger for writing functional acceptance tests in plain english for Swing GUI applications. Swinger uses Netbeans' Jemmy library under the hood to drive the app.

Cucumber allows you to write tests like this:

 Scenario: Dialog manipulation
    Given the frame "SwingSet" is visible
    When I click the menu "File/About"
    Then I should see the dialog "About Swing!"
    When I click the button "OK"
    Then I should not see the dialog "About Swing!"

Take a look at this Swinger video demo to see it in action.

Dema
ohgod, cobol for testing...
Adriano Varoli Piazza
Not so sure about that.
Dema
A: 

Window Licker for Swing & Ajax

robi