Hi,
We write a lot of integration tests that work quite well for us, we did attempt to introduce unit testing but found it more difficult to introduce as it took longer and the benefits were not that apparent.
We still have testers manually going through test scripts. We still have a long way to go to automate our testing process and we wanted to know how others approach it and what tools you use?
Thanks,
B
Edit (because someone voted this down)
This answer is about one part of your question: tools that people use. My link provides information about testing tools teams use to automate their product testing.
Check this search on Stackoverflow:
Questions tagged Automation & Testing
A compromise you may look at with the manual testers is finding a way to ouptut data results and compare them across builds.
For example pass in x set of data and you know you get y output. If you do that against the last production build and then the latest build. And look for the output is the same.
Unit tests will get you a couple thing that you may not have currently. Things in addition to tester sooner, more often, and more extensively
-Will force you to modularize and decouple code For example If your code mixes GUI and data layers you have to decouple to unit tests. But after you do that you can take your various pieces and reuse them in different projects.
-May help document your application logic. What I mean is if, your app has code that takes some data x and output data y. You can write and document the use cases in unit tests. So you will know when some x comes in what y goes out. When new developers come onto the project they can look at working unit tests (Even if the specification exists will it be correct?)
"took longer and the benefits were not that apparent"
Always the case. Quality appears to cost money up front.
"We still have testers manually going through test scripts"
And folks claim this is cheaper? How or why?
"approach it and what tools you use?"
Automate everything. It's not negotiable. Management pressure to reduce time invested into testing is something I always translate to "you mean reduce quality so we can spend more time doing rework? I can't. Find someone else who likes to do rework. I hate rework because it's always more expensive and complicated than doing it closer to right up front."
It's very difficult to retrofit unit-tests into legacy code (that is code without unit-tests). There is a book about this: Working effectively on legacy code. Unit-testing should be applied when developing new code.
It's not easy to recommend tools without information on the type of the product that is being testing. You might have to add internal interfaces so that test tools can poke the product: to test a GUI-based application, you can add an undocumented option to supply a command file that will simulate user actions.
Look at the more tedious and error-prone tasks of the manual testing, what testers complain the most of.
Automate your tests steps by steps, for instance, tests execution first and results verification (PASS/FAIL criteria) in a second phase.
Keep hard tasks manual (e.g. installation, configuration) in the beginning.
In short, apply a low hanging fruit strategy.
Also strive to reproduce every problem fixed in the codebase with an automatic test, this will help verification on the fix and constitute automatic regression testing.
Scripting (bash, Perl, Powershell, whatever) is certainly helpful when dealing with automation.
Hey.
As everyone said you need unit tests. Start with writing unit tests for new stuff. TDD would be good approach here ;).
As for manual testers, if you have a lot of them, I would try to switch for automated functional testing. Slowly but constantly convert all manual tests into automated tests.
Leave few (best) manual testers to do exploratory testing. Make them focus especially on new stuff as everything else should have automated suits prepared.
So you will have automated units test (continuous integration, you now?), automated functional testing (regression and stuff), and manual exploratory testing for new stuff (until features are stable enough to create automated suits for them).
As for tools.
For unit tests it depends on technology you use.
As for functional HP QTP (or IBM Rational Robot). It may be wise to get HP Quality Center to support testing management (or IBM tool set if you wish).
As for exploratory testing TestExplorer (it can communicate with HP Quality Center), but simple recording tools may be used too.
In your scenario, a good strategy would be to apply 'Cost Vs Benefit' analysis. For e.g. a code which has a complex logic could be the right candidate for unit testing while the code which just interacts with an external web service or database would not be the worth for doing unit testing. This is just an example. The call you have to make could be contextual.
Doing unit testing for complex logical piece of code would bring more value than doing integration testing for non-logical components. Additionally, the integration testing is a more involved activity then unit testing. While adopting to the current trends, the below order is suggested:
unit testing (automated) -> integration testing (automated) -> system testing (automated) --> functional testing (manual)