Reducing the pain writing integration and system tests

Question

I would like to make integration tests and system tests for my applications but producing good integration and system tests have often needed so much effort that I have not bothered. The few times I tried, I wrote custom, application-specific test harnesses, which felt like re-inventing the wheel each time. I wonder if this is the wrong approach. Is there a "standard" approach to integration and full system testing?

EDIT: To clarify, it's automated tests, for desktop and web applications. Ideally a complete test suite that exercises the full functionality of the application.

Answer 1

If by "make integration tests and system tests" you mean automated tests, then the answer is no, there is no standard approach. What approach choose will depend on:

• the characteristics of the application (e.g. does it have a GUI?, is it read only?, how many external dependencies does it have, etc)
• what you are trying to test (maybe you only need GUI tests, or perhaps the opposite is true and you don't really care about the GUI but the internal logic is critical)
• how quickly you want to see the results (e.g. the more you stub out the faster your tests become)
• the skillsets on your team

Personally, I love any approach that integrates with JUnit. JUnit is a fantastic framework which is well supported and easily tied into a continuous intergration server. Here are a few possible approaches:

• Selenium with JUnit - Fantastic tool for driving web applications
• Concordion - for all application types. Integrates with JUnit and allows plain english specifications of tests. Your 'fixture'/test code will hook into key words in the specification and assert or perform actions on them.
• FEST - for swing applications, again it integrates with JUnit (see a theme yet? ;) (more choices here)

The above examples provide a tremendous amount of out of the box help for testing. Of course, they still require effort to wire to your application and maintain, but the advantages are well worth it. In addition to the above, you may need to be thinking about how to stub or mock out areas of your application. Perhaps you want to do all of your testing "under the GUI" or "above the database". In the first scenario, you'll need your tests to start at the points in your code where the GUI would interact with it and in the latter you'll need to stub out the services which interact with your database.

Point being, there's lots of ways to do this. Best start with a very clear understanding of what you want to get out of your testing. Then learn what existing frameworks are out there to help you based on what you want to test, and finally, don't try to conquer the world in a night. Start small getting a few tests running. Get the green bar (always a joy to see!) Get a stable proven platform for testing and make sure you're happy with it. Then add more as you go along.