4 major OS versions: 1.5, 1.6, 2.1 and 2.2 (I guess we can safely ignore 1.0 and 2.0 now)
3 screen densities: ldpi, mdpi, hdpi
3 screen sizes: small, normal, large
Lots of system testing scenarios.
Is there are way to optimize the testing effort for an app that has to target all Android devices? What's the minimum number of emulators that are needed to cover the most cases?
Dealing with this lately and I'm very interested to know how others are doing their Android system testing.
Hey.
When optimizing testing effort, you must know that there is no magic technique, that will allow you to test less than all combinations and be 100% that it works for every combination.
Now, there are techniques that let you lower test effort in exchange for risk that in some particular combinations (you are not checking) it may not work.
Those techniques are Pairwise testing and Orthogonal Array. You may google for this, but if you want quick version go to this link http://www.developsense.com/pairwiseTesting.html and look at tables on the bottom:
Table 10: All Combinations for Three Variables of Three Levels Each - this one contains all 27 combinations.
Table 11: All-Pairs Array, Three Variables of Three Levels Each - this one contains reduced number of combinations - 9 combinations to be tested.
Just match A/B/C to OS/densities/size and for each of them R/G/B to appropriate values for given variable.
I am only Testing on 1.5 and 2.2 - If it works at the two extremes I expect to work in the middle. Same for screen sizes.