How to break someone into testing?

Question

OK. Our product works. Beta testers are actually getting their stuff done. Time for the next iteration. But how to ensure quality? We need a tester!

How do I get someone fresh off the street started in testing? I have no clue on how to do it myself (I'm a developer, not a tester)!

We are a tiny team:

• 2 architects (as in buildings, not software, they are the domain experts here) figuring out what to build
• me building it
• and a new guy to do some testing before we push releases out

None of us has a clue on how to do this professionally. So far we have:

• a bunch of virtual machines spanning the configurations we would like to test
  • various versions of windows
  • german and english, the two languages likely to be in use by our customers
  • the host software we are writing for (Autodesk Revit Architecture 2010, we are building a plugin for energy calculations)
• a text document describing some tests I did (installed release xyz, did this, did that, etc.)
• a bug tracking system the tester can add all the bugs he finds

I expect we will need a test script. But how? Who? What? When?

Answer 1

Well, are you looking to expand your team with a tester? Have you considered just hiring a test specialist from a consultancy firm?

Answer 2

Why are you looking for "someone off the street"? To me, it sounds kind of like asking "I want to hire a new programmer, how do I get someone off the street and get him up to speed programming my software?". Why would you want to do that, over hiring someone who is a programmer already?

In your situation, which is that you don't know much about testing, I'd definitely think about hiring someone with experience in the field.

Specifically, I'd probably look for:

1. Someone with some experience performing tests under his belt (since you're going to want him actually doing tests).
2. Someone with some experience writing test plans/etc.
3. Someone with some experience running a QA team.

The last point is optional, but hopefully your team will be growing as your software grows, so it might make sense to get someone who can grow in the role as well (not to mention having the experience to help you decide when and how to grow the QA team).

Answer 3

Ideally, you'll need to give the tester:

• training to make sure he knows the product to be tested.
• documentation on what the expected results are.
• test plans - what needs to be tested and how
• a test tracking system to track what is being tested, what passed the tests, what needs to be fixed, etc. That system does not have to be too sophisticated, depending on the size of the project, an Excel spreadsheet may suffice.

Answer 4

In their podcast #64, Jeff and Joel discuss (among other things) what skills a good tester should possess. Transcript also available (about halfway down the page)

Answer 5

Before you get somebody to test, make sure you meet the requirements for testing. At a minimum you need:

A specification: Some authoritative source on what the application is supposed to do. This could be an expert that can answer any and all questions on exactly what the app is supposed to do, but the more that is written down and the more formally defined it is the better.

Time: Testing takes time. You can't hand off an application to the tester 30 minutes before it's supposed to go live and expect any worthwhile results. If you're doing waterfall development, testing will require a lot of time at the end. Lots of other development models let testing run in parallel with development, which saves a lot of time, but regardless of the model you use, testing will require more time than not testing.

If you don't have these two things, quality assurance is just a pipe dream.

Now if you do have those met, and you're trying to train somebody to test, here's my crash course on testing.

Fundamentally, testing an application means that you are attempting to ensure two things:

• The program does what it is supposed to do.

• The program does not do what it is not supposed to do.

That's the core mindset that I use. Building from that I approach things in terms of actions and attempt to verify:

1. An expected action with expected preconditions produces an expected effect.
2. An expected action with unexpected preconditions produces no effect or is handled appropriately.
3. An unexpected action produces no effect or is handled appropriately.
4. No unexpected effects occur.

Item 1 comes directly from the spec: You make sure that the program does what it is supposed to do.

Items 2 and 3 are where the art of testing comes in. What unexpected actions and preconditions can I perform? I could try to enter the wrong password. I could try to directly type in the URL of a supposedly secured page. I could try to paste odd unicode characters into a text field. I could try to put SQL or javascript code into a text field.

Item 4 is the infinite no-man's land of testing, the part that makes complete testing impossible. (2 and 3 are also infinite, but not as depressing to think about.) That doesn't mean you ignore it. You always keep an eye out for anything unusual. Also, sometimes inspiration strikes and you think of a possible way to cause an unexpected effect: "What happens if I log in between 11:59:59PM and 12:00:00AM on the third tuesday of the month? Oh look, it made me an administrator." Technical knowledge and a peek inside the black box help with coming up with scenarios like that.

There is a whole lot more to say about testing, but that's the bare minimum I can think of: The technical requirements and the approach to the problem.