How can I write an automated unit test of a GUI in Xcode?

Question

I want to write a unit test of just the GUI part of my Cocoa application.

In the textbook unit test, there's a test framework and test case that calls the unit under test. All the code below that unit is mocked out. So, both the input and the output are controlled and monitored; only the code in the unit under test is tested.

I want to do the same thing where the unit under test is my GUI:
1) Set up some kind of framework where I can write code that will manipulate and inspect GUI controls.
2) Connect my GUI controls to mocks of my actual code, not to the real instances.
3) Run the test, which manipulates the controls and then checks the mock object to see whether the correct methods were called with the correct parameters and checks the GUI to see whether the responses from the mock object causes the correct changes in the widgets.

Anyone doing this? If so, how? Any ideas on how I could do this?

Thanks,

Pat

(Edit) To give a very specific example, I want to:
1) Write a test case that will select the menu item 'MyMenu' -> 'MyItem'. In this test case, I want to check to see that the method [AppDelegate doMyItem] gets called precisely once and that no other methods in AppDelegate get called.
2) Generate a mock object of AppDelegate. (I know how to do this)
3) Somehow (handwaving here) link my application so that a mock instance of AppDelegate is linked in instead of the real one.
4) Run the test. Watch it fail because 1) I haven't created MyMenu yet. 2) I haven't created MyItem yet. 3) I haven't done the IB work to connect MyItem to [AppDelegate doMyItem], or 4) because I haven't written the 'doMyItem' method yet.
5) Fix the above four issues (one at a time if I'm feeling really pedantic that day).
6) Run the test again and watch it succeed.

Does this make the question clear?

Answer 1

Here are a couple of popular ways of doing this in general (should work with most if not all cocoa compatible languages).

1 - create a callback interface. One of the inputs when creating your GUI elements is an implementation of this interface. When there's a user interaction, the GUI element calls an update function on that interface. Have a real implementation and a test implementation.

2 - Use event-handlers. Register all of your GUI elements with one or more event-handlers, and have the GUI generate events on user interaction. Have an event handler interface with two implementations, again one for real use and one for testing.

Edit: whoops, missed requirement #1. Never done this with OSX specific controls, but in general there are two approaches.

1 - create a script or app that generates user-like input. Has the drawback of not being easy to actually inspect the GUI. You instead need to generate good test cases to make sure that everything that should be there is, and nothing extra is there.

2 - create an interface with a test implementation that replaces the rendering and interface layer. This is easier with libraries like SDL or directFB and less so with with things like the OSX API, win32 API, etc.

Edit: responding to edit in question.

In the case of your example, using a seperate testing app and event handlers here's how it'd look:

Your test application is a simple app or script that starts up your GUI and then generates mouse / keyboard events based on input files. As I've said, never done this in OSX (only QNX). With any luck you'll be able to generate mouse and keyboard events with the API, but you'll have to ask someone else if it's possible.

So create an input for your test-case. The test app will parse this to know what to do. It may be simple XML like this:

<testcase name="blah"><mouseevent x="120" y="175" type="click"/></testcase>

or whatever the mouse sequence may actually be.

When your script executes that command it will click the mouse on that button. Your event handler will pick up on this. But now you should be running your app with a --test flag or somesuch so that it's actually using the test event handler. Instead of doing whatever your app normally does, the test event handler can do some custom action. For instance it may do some of the normal actions (you still need the GUI to respond) and then send a message (via socket, pipe, whatever) to your test app.

Your test app will pick up this message and compare it to what it expects to see. So now maybe your testcase XML looks like this:

<testcase name="blah">
   <mouseevent x="120" y="175" type="click"/>
   <response>doMyItem() called</response>
</testcase>

If the response generated from the event handler is different, then the test case has failed. You can print out the actual response to help in debugging.

Answer 2

Two principles, two links:

• Make the view as dumb as possible, with the passive view pattern: this makes GUI easier to test
• Trust but verify: Trust Cocoa implementation of buttons, menus, ... But verify that target and action are correctly connected, that bindings are as expected.

Answer 3

Have you looked into the accessibility framework? It should let one application inspect the UI of another application and generate user-like interaction events.

Accessibility Overview