Test plans and how best to write them

Question

We're trying to figure out the best way to write tests in our test plan. Specifically, when writing a test that is meant to be used by anyone including QA staff, should the steps in the test be very specific or more broad giving the tester more leeway in how the task can be accomplished. As a very simple example, if you're testing opening a document in word processing document, should the test read:

1. Using the mouse, open the file menu
2. Choose "Open File..." in the file menu
3. In the open file dialog that appears, navigate to x and double-click the document called y

OR

1. Bring up the file open dialog
2. Open the file y

Now I realize one answer is probably going to be "it depends on what you're trying to test" but I'm trying to answer a broader question here: If the test steps are too specific do we risk a) making the testing process to laborious and tedious and more importantly b) do we risk missing something because we wrote down too specific a path to achieve a goal. Alternatively, if we make it broad do we depend too much on the whims of the tester at the time and lose crucial testing of paths that are more common to customers/clients?

Answer 1

I'm not a tester but in my opinion it is vital to document the "UI route" that the test must take to avoid any confusion.

I have worked on countless defects that I could not reproduce simply because I was not accessing the fuction from the same UI path as the tester was. e.g. Right Click menu vs Toolbar or functions that can be carried out from various dialogs, etc etc..

Answer 2

My first question would be - why isn't your QA department writing the test plans? Typically, software developers provide QA with a functional spec of how the software is supposed to work and then QA creates test plans based on that.

With that said, I would suggest being very specific with the steps since you are detailing how things are supposed to work. It is then the job of the tester to make sure that your specific steps get the desired results and it is also their job to deviate from the plan and try to break things.

If there are multiple ways to achieve a goal, you need to describe each path to get there.

Answer 3

It sounds like your QA staff is really QC (Quality Control) if they are not responsible for writing tests. If they actually are responsible for writing tests, your tests will be helpful, but specifications that are clear would be a better source for them to write the tests themselves. Even better would be to have them as part of the review process for the specs so that they can ask for details that will allow them to write tests.

If you really are in a position where you are writing tests for other people, there are some considerations. You will want a painful level of detail if :

• the people running the tests are not able to come ask you questions
• the people running the tests are not familiar with your product

You can avoid some details if these are not true. However, it still depends :)

This all being said, what you will have written is not what is generally considered a 'test plan'. A Test Plan describes what types of tests will be executed (functional, regression, security, etc.), what features are to be tested, maybe even what tests are to be written, who will be doing the testing, when groups of tests are scheduled and other planning type activities.

What you describe above is simply a set of tests.

Answer 4

Let's distinguish Test Plan and Test Suites :)

Test Suite is set of tests itself

Test Plan is [part of] Test Suite + available resources (people, hardware, time, ...).

It's OK to have both variants (detailed and "rough") described in your test documentation, just place detailed and "rough" tests to different documents and prioritize these documents.

Then, when you have enough time to test product completely, you take all documents of, say, category A and test product in accordance with these documents. If you have no time, but need quick conclusion about quality, you take category B documents and pass checks described there.

good side: you can select how to test product

bad side: you need "duplicate" documents

Answer 5

The first is feature testing. Test with detailed steps containing UI route as there is possibly more routes than one to the destination. Test all routes. The latter sounds more like usability testing. It should be done too but not only by your testers but also by external people.

Answer 6

theres pros and cons to treating your tester 'like a retard' (i.e. as though they have no knowledge of the system or computers in general)

if you spell things out in minute detail (e.g. "from the file menu, select 'Open'...") than the benefit is you can use contractors that arent familiar with your system. but it takes you longer to write like this

if you skip a lot of detail (e.g. "open a document file..."), than whoever uses your test plan is more likely to get stuck, and than interrupt you for clarification. but it is much faster to write

it can be a false economy to think its faster if you do the brisk version, if you end up just spending extra time explaining the system to the qa person

i have an article where i go into more depth on this topic: Writing a System Test Plan

in this article, i favour the more detailed approach. but i have been developing a 'mid-point' between these two approaches lately (called a FEATURE test plan), but im not at a point where its mature enough to share yet

-- LM

Answer 7

I would just direct you to the right article here: http://developergeeks.com/article/1/guidelines-for-testing-and-test-planning