Hey In my workplace we use scenario based testing. However whenever somethign is fixed or a new patch is added new scenarios are added as a result the list keeps getting longer and longer and takes 3 days plus to test the application. Is there a way to do proper testing without taking a long long time? What do you use?
Thanks
Hi Andreas
Only 3 days to test your application ! We've got test jobs that run for maybe 15 days. And I guess other lurkers around here can tell you that they have even bigger test jobs; you know the drill -- when I were a lad we didn't even have a hole in't' road to live in.
But seriously, 3 days to fully test a release candidate with a benefit stream worth O(USD10^7) doesn't seem outrageous to me. On the other hand, if it's taking you 3 days to test changing one field on a GUI from 12 characters to 24 characters, then that does seem a bit too much. I think your question might be better phrased as 'How much of our development time should be spent on testing ?' and the answer might be anything from 10% -- 50% (possibly higher for safety-critical systems). If you are spending 2 days developing a patch, then testing should probably take no more than 1/2 day.
And yes, the scenario where as your application expands your test suite expands too, is very familiar. However if we add a new bit of functionality we tend to add new tests; a better approach, -- one we never have time for though we always have time to deal with not taking it -- is to modify existing tests. Modify code -> modify tests; add new code -> add new tests.
Yes, we use automated testing as much as we can; we use a lash-up of bash scripts, python programs and make to drive our automated tests. The processors we use never complain that testing is boring and repetitive, so we have no ethical qualms about working the poor dawgs close to heat-death. Sadly local labour laws prevent the same robust management principles being applied to the carbon-based life forms in our offices.
Regards
Mark