How will you categorise the various software testing methods?

Question

Would this be right??

1. Black Box

   1.1 Functional

       1.1.1 Equivalence
       1.1.2 BVA
       1.1.3 Use case
       1.1.4 Regression
       1.1.5 UAT
   

   1.2 Non Functional

       1.2.1 Testing the System Design
   

2. White box

   2.1. Functional

          2.1.1 Unit
          2.1.2 Integration
          2.1.3 System
   

Do the above fall under the right categories?

The reason I ask this is because as a part of a report I was trying to come up with a good reference that categorised Test Techniques well. This is what my analysis and research from various sources gave me. And I hope this is helpful for someone else who might be doing the same research, but if its incorrect it should be updated.

Answer 1

You might also consider the case when several programs depending one on another are developed simultaneously. You have then to take into account the applicative architecture which groups all those applications into several functional domains

So, for instance, a financial application having to process a large number of data would be one functional domain, in which you would have to develop a:

• dispatcher module in order to process those data on several computers
• GUI in order to see what is going on
• launcher in order to initiate the right connections retrieve the correct data and format them
• and so on

But that would only be one functional domain, as others would have to be developed in order to exploit the results of your programs (for instance, a "referential domain" would be there to store those results into various databases, and offer a communication bus for other programs to access them: that would be a second functional domain).

So I would add to your tests the following categories:

• Assembly testing: when you test within your own functional domain (on an assembly server when you deploy the different applications of your domain, with a set of testing data)
• Integration testing: when you test all the applications from all the functional domains, which is also called front-to-end testing.

Note: "integration testing" is not the same as "continuous integration testing", which basically can process the black and white tests you describe, for one program, on a very regular basis.

The tests I am referring to are executed a few times a week by an:

• "Project Operational Architecture" team of your domain for assembly tests: usually some developers of your team which set up an assembly server, check if the data are up-to-date and deploy the various program you are in charge to develop.
• "Production Operational Architectural" team, in charge of setting a "production-like" environment and who is the only one able to really test the all chain of application from font to back.

Note: an "Operation Architecture" team has the role to "make operational an execution environment", meaning to have:

• the right logistic team contacts in order to have the right servers and networks,
• the right application teams contacts in order to know about the various start/stop application processes and deployment procedures of all the application of your system!

In short: your categories are for one program, but when you are developing an IS (Information System), you are forced to acknowledge the fact that you are not talking about "one exe developed by one team deployed on one production machine"... and then, welcome to an all new world of testing ;)

Answer 2

I think that your categorisation is a good first step.

Separation between black box and white box (some prefer glass box) testing focuses on whether you have access only to the specification or more (design, source code).

I would add a second separation between functional and structural testing, which focuses on whether you want to consider what the software does (functional) or how it does it (structural).

A third separation deals with how you generate test inputs, deterministically or statistically (with a deliberate distribution and not randomly). Either way, your focus is on what coverage you target.

Finally a well known separation is between different levels of software cycle: unit testing, integration, system, acceptance, ...