views:

74

answers:

6

I will be testing an application that claims to run on Windows XP, Windows Vista, and Windows 7. I know that I have to test the application for each of those on 32-bit and 64-bit (if it exists), but I'm not sure if it's necessary to test each edition of each operating system.

For example, do I really need to test for Windows 7 Starter if I already tested Windows 7 Ultimate or Home? Do I need to test Windows Vista Business if I already tested Windows Vista Home Premium? And so on ...

It would be very time-consuming to test each and every edition of each of the three flavors that are supposed to be supported:

  • Windows XP Home w/ SP2
  • Windows XP Professional w/ SP2

  • Windows Vista Starter

  • Windows Vista Home Basic
  • Windows Vista Home Premium
  • Windows Vista Business
  • Windows Vista Enterprise
  • Windows Vista Ultimate

  • Windows 7 Starter

  • Windows 7 Home Basic
  • Windows 7 Home Premium
  • Windows 7 Professional
  • Windows 7 Enterprise
  • Windows 7 Ultimate

Let me know what you think.

+1  A: 

Hey,

It would require analysis from your part.

If you'd use dll's or features only available in some Windows editions, or rely on Hotfixes integrated in an SP but not in the base edition, you'd have to test on each of the different versions.

However, I'd say in most situations testing what you suggested should suffice.

I would definitely test the following versions from your list:

  • Windows XP Home w/ SP2 [Yes]
  • Windows XP Professional w/ SP2 [No]

  • Windows Vista Starter [Yes]

  • Windows Vista Home Basic [No]

  • Windows Vista Home Premium [No]
  • Windows Vista Business [No]
  • Windows Vista Enterprise [No]
  • Windows Vista Ultimate [No]

  • Windows 7 Starter [Yes]

  • Windows 7 Home Basic [No]

  • Windows 7 Home Premium [No]
  • Windows 7 Professional [No]
  • Windows 7 Enterprise [No]
  • Windows 7 Ultimate [No]

My understanding is: If it works on the most stripped basic version, it will work on the the ones that are more feature packed.

Trefex
+1  A: 

The only difference between Home/Professional/Ultimate/Orgasmic would be the larger scale features that are included. The API should be identical.

Mark Ransom
Do you have any examples of the larger scale features?
Brian T Hannan
Connect to company networks easily and more securely with Domain Join.That would be my example...
Trefex
@Brian: XP Pro includes Remote Data Connection (RDP). Windows 7 Pro includes Virtual XP mode. I'm sure Microsoft has charts that identify all the differences between the versions.
Mark Ransom
Vista features compared between editions: http://windows.microsoft.com/en-US/windows-vista/products/compare
JYelton
A: 

A possible alternative is to skip testing and simply ship it. This provides you with a small army of testers with all kinds of environments, some of whom will report to you (with angry e-mails or demands for refunds, perhaps).

For some years, I was in a happy position where I had essentially a captive customer base (this was in-company) and my code either worked for them or they reported on failure and I'd fix it the same day. THEY WERE GRATEFUL to get fixes so quickly; other departments tested their code and had lengthy release cycles but not really fewer bugs; and if something went wrong it took ages to fix.

This may not be good advice for you, but (as my anecdote shows) it would depend on the circumstances. If you're not worried about annoying a few users or tarnishing your reputation, this might allow you to cut a corner or two.


EDIT:

Based on your feedback, you're in the (more common) situation where this kind of operation would earn you a spanking. In that case, I'd agree with the recommendations of some other answers that encourage you to test all major releases but not the different editions.

As the programmer, you should theoretically have a feel for what distinguishes the Home/Premium/Whatever versions. These days, as far as I've seen, that tends to be stuff like:

  • Is there server-type software (like IIS) included?
  • Is there a limit on network connections?
  • Does it ship with all languages?
  • Which multimedia gadgets and codecs does it include?

In some cases, the difference in versions came down to a single flag stored in some number on the CD... identical code, just an option for toggling capabilities. Unless the subset of capabilities has an impact on your code, you're probably safe to ignore it. A sensible approach might be to test with the lowest common denominator (i.e. the "poor peoples' editions") of the product, as that will

  • be the least expensive to purchase for testing on; and
  • have the greatest number of restrictions on what your code can do.

But as usual, Your Mileage May Vary.

Carl Smotricz
You worked for Windows? :D
Trefex
Customers who don't realize they are beta testers are usually unhappy customers; unless you are lucky and the software is mostly devoid of bugs. "...skip testing and simply ship it." -- sounds like the practice of several game development companies... none of which are still in business. Wonder why?
JYelton
Yes, I agree that you can do that for in-house customers but the customers here are spread across the globe and they are paying customers who might not like buying a buggy piece of software.
Brian T Hannan
@Trefex: I told you, this was an in-house setup. Far more coding is written and used inside companies than goes on the shelves for shrink-wrapping to the public. @JYelton: I carefully explained circumstances where this approach could work but was sure to point out that it's generally a bad idea.
Carl Smotricz
@Brian That certainly narrows your choices down a bit! I'll throw an update into my probably-doomed answer for you...
Carl Smotricz
@ Carl just teasing mate :)
Trefex
@Carl: BTW ... I'm working as the tester on this application ... not the developer so I don't necessarily know what distinguishes Home/Premium/Whatever versions.
Brian T Hannan
+2  A: 

It would be good, but it is not necessary. A prudent approach would be to test it on the edge cases.

For example, if it works on XP SP1 and SP3, it probably works on SP2. If it works on Windows 7 Home Basic and Windows 7 Ultimate, then it probably works on Windows 7 Professional.

Use your best judgment. Its easy to post an answer that says, "Do Everything!", but in reality, that is not practical.

Stargazer712
+1: edge-case testing and deductive reasoning is a great approach
JYelton
+5  A: 

In our workflow we test each platform and architecture, not every edition.

For example, test on XP, Vista, and 7, and 32- and 64- bit architectures. There is no need to test both XP Home and Professional on the same architecture unless some feature of your software depends on a functionality only present in Professional.

JYelton
I agree; testing each edition sounds wasteful in the general case.
Carl Smotricz
A: 

I'd start with the Windows Application Compatibility Toolkit from Microsoft, the Win 7 version is here. You should be able to back track from there through the Vista and XP ACTs.

Lazarus